Image Processing — Simple Homography

Homography Matrix

Have you ever wanted to know what would be the top view of a static image?

In image processing, we can perform the different transformations of images depending on our needs.

One of the sought after techniques are called Homography Matrix Transformation.

The homography technique is an image processing technique where we perform mapping of an image between two planar projections. It can simply be represented by a 3x3 Matrix.

You can perform different transformations using tomography such as perspective change, rotation, skew change, and the likes.

To better understand it, let us show some examples:

Let us load some sample images.

import numpy as np
from skimage.io import imshow, imread
from skimage.color import rgb2gray
import matplotlib.pyplot as plt
sample = imread('food3.jpg')
sample_g = rgb2gray(sample)
fig, ax = plt.subplots(1,2,figsize=(15,10))
ax[0].imshow(sample)
ax[1].imshow(sample_g,cmap='gray')
ax[0].set_title('Colored Image',fontsize=15)
ax[1].set_title('Grayscale Image',fontsize=15)
plt.show()

Figure 1: Sample Image (Image by Author)

Notice that our sample image is a food plate which in a different perspective. Using the Homography matrix, we will try to show how the top view of the image.

To start, we would know the different corners of the plate, we can simply have a trial and error scheme to determine the corners.

fig, ax = plt.subplots(1,2,figsize=(15,15))
ax[0].imshow(sample)
ax[1].imshow(sample)
ax[1].add_patch(plt.Circle((700,10),20,color='r'))
ax[1].add_patch(plt.Circle((10,340),20,color='r'))
ax[1].add_patch(plt.Circle((1300,400),20,color='r'))
ax[1].add_patch(plt.Circle((490,1000),20,color='r'))
ax[0].set_title('Colored Image',fontsize=15)
ax[1].set_title('With Label on Corners',fontsize=15)
plt.tight_layout()
plt.show()

Figure 2: Image with Markings (Image by Author)

The red dots are the markings that correspond to the corners of the plate. Since we know already the coordinates of these corners, we would be putting them in an array since we need it for the transformation.

src = np.array([700,10,
                10,340,
                500,1010,
                1300,400]).reshape((4, 2))

We would be also initializing an array for the coordinates where we will map the coordinates of the corners of the plate. The sample array is below.

dst = np.array([100, 100,
                100, 1000,
                1000, 1000,
                1000, 10,
]).reshape((4, 2))

Scikit-Image Library has already a predefined function for the transformation of the homography matrix. We will be using for easier flow:

from skimage import transform
#transform 
sample_tf = transform.estimate_transform('projective', src, dst)
sample_tf1 = transform.warp(sample, tform.inverse)
fig, ax = plt.subplots(1,2,figsize=(15,15))
ax[0].imshow(sample)
ax[1].imshow(sample_tf1)
ax[0].set_title('Colored Image',fontsize=15)
ax[1].set_title('Projected Image',fontsize=15)
plt.tight_layout()
plt.show()

Figure 3: Projected Image (Image by Author)

From Figure 3, we can see that we can plot the top view of the image. Some interesting insights we can see from here are:

1. Depending on the Initialized Corners of the Source Image, the resulting project image will leave out the pixel values that are within the bounding box of corners.
2. The perspective of the objects will be transformed also along with the project. Notice that the dessert cups are projected more elongated that the source image.
3. Since you are projecting the source image in to a different planar matrix, there will be possibility of black pixels for some of the parts in resulting projected image.

Summary

For this Article, we were able to show how we can use a simple image processing technique called transformation using Homography Matrix. Using such technique will alow to be able to rotate, resize and even change the perspective of our image.

Stay tuned for more articles!