Image Enhancement (Part 1)
Enhancing Images at max!
Considering that we can already read our digital images as Numpy Arrays and we can already convert to different kinds of color channels, what can we do next?
The answer is enhancement!
Normally, Digital Images was crystal clear but this does not happen all the time. Sometimes images would have certain noises included in the image file like for example a rain filter of the sort. Sometimes images would appear to be too bright or too dark due to their background. Different images can be enhanced by using different image enhancement techniques.
Let us Focus First on Fourier Transform Image Enhancement!
Let’s Go!
Fourier Transform
The goal of Fourier Transform is to show the images as a form of waves and periods. From this period and waves, we can actually transform them into their periodic form wherein only the DC Value would be shown. Remember that Fourier Sign Wave can be transformed into a Periodic Form where the Highest Value would e DC Value where all the needed information is located. Normally, a perfect sign wave would only a DC Value, but in this imperfect world, we would always see some spikes on the sign wave, and when we transform it to its counterpart, we can see small values on its harmonic frequencies. These small values are typically called noises. We usually don’t want them in any waveform.
In terms of images, when we see such noises, we try to mask them or initialized them with another value. The utmost important should be only in the central DC value. Some examples of images with noise are as follows, and we will try to take the noise in the image.
First Lets us read the Image:
from skimage.color import rgb2gray
from skimage.io import imread,imshow
import matplotlib.pyplot as plt
import numpy as npsample = rgb2gray(imread('lunar_orbiter.jpg'))#a nifty function to show the Fourier Transform of an Array
sample_f = np.fft.fftshift(np.fft.fft2(sample))fig, ax = plt.subplots(1,2,figsize=(12,5))
ax[0].imshow(sample,cmap='gray')
ax[0].set_title('Original',fontsize=15)
ax[1].imshow(np.log(abs(sample_f)),cmap='gray')
ax[1].set_title('Fourier Transform',fontsize=15)
plt.tight_layout()
plt.show()
As you can see on the sample image, it has a sort of horizontal white lines on its surface. The image is actually an image from a satellite camera, the white lines form square-like features on the image which is seen as noise. The image would look much better if the white line is gone right?
The Fourier Transform plot shows the DC Value which is normally at the center of the graph. YOu can see that it has the highest value since it is the brightest. A perfect Fourier Transform should only have the DC value at the center, so the other white dots seen on the horizontal and vertical axis should be masked. The code below shows the masking:
sample_fa = sample_f.copy()
sample_fa.shape(574,548)
sample_fa[:280,sample_f.shape[1]//2] = 1
sample_fa[290:,sample_f.shape[1]//2] = 1
sample_fa[sample_f.shape[0]//2,:sample_f.shape[1]//2] = 1
sample_fa[sample_f.shape[0]//2,1+sample_f.shape[1]//2:] = 1fig, ax = plt.subplots(1,2,figsize=(12,5))
ax[0].imshow(sample,cmap='gray')
ax[0].set_title('Original',fontsize=15)
ax[1].imshow(np.log(abs(sample_fa)),cmap='gray')
ax[1].set_title('Fourier Transform with Mask',fontsize=15)
plt.tight_layout()
plt.show()
As you can see in the figure we have inputted a mask of 1 value along the cross-segment of the Fourier transform. We have inputted the mask on these specific locations since it has the brightest value among the other Fourier Transform values. This means that these parts are the noises of the image.
fig, ax = plt.subplots(1,2,figsize=(12,5))
ax[0].imshow(sample,cmap='gray')
ax[0].set_title('Original',fontsize=15)
ax[1].imshow(abs(np.fft.ifft2(sample_fa)), cmap='gray')
ax[1].set_title('Enhanced',fontsize=15)
plt.tight_layout()
plt.show()
You can see the on figure, the original, and the enhanced image. We can note that the cross horizontal white lines are have disappeared. This is due to masking that we have done
Let us try another example!
sample2 = rgb2gray(imread('moon_image.png'))#a nifty function to show the Fourier Transform of an Array
sample_f2 = np.fft.fftshift(np.fft.fft2(sample2))
fig, ax = plt.subplots(1,2,figsize=(12,5))
ax[0].imshow(sample2,cmap='gray')
ax[0].set_title('Original',fontsize=15)
ax[1].imshow(np.log(abs(sample_f2)),cmap='gray')
ax[1].set_title('Fourier Transform',fontsize=15)
plt.tight_layout()
plt.show()
For this specific example, we can see that the original image has a sort of leather clothes filter on it. This shows that the image has already had different noise on it. Upon checking the Fourier Transform, we can see different bright lights that are different from out DC Value at the center. We will try to mask each of the points to properly enhance the image.
sample_f2a = sample_f2.copy()
sample_f2a[:(sample_f2.shape[0]//8),sample_f2.shape[1]//2] = 1
sample_f2a[:(sample_f2.shape[0]//8)*2,sample_f2.shape[1]//2] = 1
sample_f2a[:(sample_f2.shape[0]//8)*3,sample_f2.shape[1]//2] = 1sample_f2a[5*(sample_f2.shape[0]//8):,sample_f2.shape[1]//2] = 1sample_f2a[sample_f2.shape[0]//2,:(sample_f2.shape[1]//4)] = 1
sample_f2a[sample_f2.shape[0]//2,:(sample_f2.shape[1]//8)*3] = 1
sample_f2a[sample_f2.shape[0]//2,(sample_f2.shape[1]//8)*5:] = 1sample_f2a[:,2+(sample_f2.shape[1]//8)*2] = 1
sample_f2a[:,2+(sample_f2.shape[1]//8)*3] = 1
sample_f2a[:,4+(sample_f2.shape[1]//8)*5] = 1
sample_f2a[:,4+(sample_f2.shape[1]//8)*6] = 1
sample_f2a[1+(sample_f2.shape[0]//8)*2,:] = 1
sample_f2a[1+(sample_f2.shape[0]//8)*3,:] = 1
sample_f2a[1+(sample_f2.shape[0]//8)*5,:] = 1
sample_f2a[1+(sample_f2.shape[0]//8)*6,:] = 1
#
fig, ax = plt.subplots(1,2,figsize=(12,5))
ax[0].imshow(sample2,cmap='gray')
ax[0].set_title('Original',fontsize=15)
ax[1].imshow(np.log(abs(sample_f2a)),cmap='gray')
ax[1].set_title('Fourier Transform with Mask',fontsize=15)
plt.tight_layout()
plt.show()
We can see from the figure that we were able to mask the other DC value signal that is seen on the Fourier Transform. You can note that the center DC Value was preserved to make sure that the original DC Value of the image is preserved.
fig, ax = plt.subplots(1,2,figsize=(12,5))
ax[0].imshow(sample2,cmap='gray')
ax[0].set_title('Original',fontsize=15)
ax[1].imshow(abs(np.fft.ifft2(sample_f2a)), cmap='gray')
ax[1].set_title('Enhanced',fontsize=15)
plt.tight_layout()
plt.show()
We can compare the original sample versus the enhanced version, we can see a big difference! The enhanced image is also ready very clear without the sort of leather cloth filter. The image looks crisper without the sort of dots filter.
What if we use an image without noise?
Let us try our flower bouquet!
sample3= rgb2gray(imread('flower.jpg'))#a nifty function to show the Fourier Transform of an Array
sample_f3 = np.fft.fftshift(np.fft.fft2(sample3))
fig, ax = plt.subplots(1,2,figsize=(12,5))
ax[0].imshow(sample3,cmap='gray')
ax[0].set_title('Original',fontsize=15)
ax[1].imshow(np.log(abs(sample_f3)),cmap='gray')
ax[1].set_title('Fourier Transform',fontsize=15)
plt.tight_layout()
plt.show()
We can see that the noise can be seen only on a cross-section from the center.
sample_f3a = sample_f3.copy()
sample_f3a[:sample_f3.shape[0]//2,sample_f3.shape[1]//2] = 1
sample_f3a[10+sample_f3.shape[0]//2:,sample_f3.shape[1]//2] = 1
sample_f3a[sample_f3.shape[0]//2,:(sample_f3.shape[1]//2)] = 1
sample_f3a[sample_f3.shape[0]//2,10+sample_f3.shape[1]//2:] = 1fig, ax = plt.subplots(1,2,figsize=(12,5))
ax[0].imshow(sample3,cmap='gray')
ax[0].set_title('Original',fontsize=15)
ax[1].imshow(np.log(abs(sample_f3a)),cmap='gray')
ax[1].set_title('Fourier Transform with Mask',fontsize=15)
plt.tight_layout()
plt.show()
fig, ax = plt.subplots(1,2,figsize=(12,5))
ax[0].imshow(sample3,cmap='gray')
ax[0].set_title('Original',fontsize=15)
ax[1].imshow(abs(np.fft.ifft2(sample_f3a)), cmap='gray')
ax[1].set_title('Enhanced',fontsize=15)
plt.tight_layout()
plt.show()
We can see from the results of the flower example, that there is no major difference. Normally it should have but for our example, we see that the only difference it makes is that it embosses the flower in the middle and also, there are sort of additional horizontal and vertical lines. This is actually due to the mask that we have inputted earlier.
Summary
The use of the Fourier Transform Image Enhancement technique can be very effective for heavily noisy images but for images that are relatively crisp or don't have any noise, it works poorly.
Stay tuned for Part 2 and Part 3!
