Image Enhancement Chapter 4
Linear noise smoothing
Linear (spatial) filtering is a function with a weighted sum of pixel values (in a
neighborhood). It is a linear operation on an image that can be used for blurring/noise
reduction. Blurring is used in pre-processing steps; for example, in the removal of small
(irrelevant) details. A few popular linear filters are the box filter and the Gaussian filter. The
filter is implemented with a small (for example, 3 x 3) kernel (mask), and the pixel values
are recomputed by sliding the mask over the input image and applying the filter function
to every possible pixel in the input image (the input image's center pixel value
corresponding to the mask is replaced by the weighted sum of pixel values, with the
weights from the mask). The box filter (also called the averaging filter), for example,
replaces each pixel with an average of its neighborhood and achieves a smoothing effect(by
removing sharp features; for example, it blurs edges, whereas spatial averaging removes
noise).
The following sections illustrate how to apply linear noise smoothing on images first using
the PIL ImageFilter module and then using the SciPy ndimage module's filter
functions.
Smoothing with PIL
The following sections demonstrate how the functions from the PIL ImageFilter module
can be used for linear noise smoothing; in other words, noise smoothing with linear filters.
Smoothing with ImageFilter.BLUR
The following shows how the PIL ImageFilter module's filter function can be used to
apply a blur to denoise a noisy image. The noise level on the input image is varied to see its
impact on the blur filter. The popular mandrill (baboon) image is used as the input image
for this example; the image is protected by a Creative Commons license (https://
creativecommons.org/licenses/by-sa/2.0/) and can be found at https://www.flickr.
com/photos/uhuru1701/2249220078 and in the SIPI image database: http://sipi.usc.edu/
database/database.php?volume=miscimage=10#top:
i = 1
pylab.figure(figsize=(10,25))
for prop_noise in np.linspace(0.05,0.3,3):
im = Image.open('../images/mandrill.jpg')
# choose 5000 random locations inside image
n = int(im.width * im.height * prop_noise)
x, y = np.random.randint(0, im.width, n), np.random.randint(0,
[ 157 ]
Image Enhancement Chapter 4
im.height, n)
for (x,y) in zip(x,y):
im.putpixel((x, y), ((0,0,0) if np.random.rand() < 0.5 else
(255,255,255))) # generate salt-and-pepper noise
im.save('../images/mandrill_spnoise_' + str(prop_noise) + '.jpg')
pylab.subplot(6,2,i), plot_image(im, 'Original Image with ' +
str(int(100*prop_noise)) + '% added noise')
i += 1
im1 = im.filter(ImageFilter.BLUR)
pylab.subplot(6,2,i), plot_image(im1, 'Blurred Image')
i += 1
pylab.show()
The following screenshot shows the output. The smoothed image quality gets poorer as the
input image gets noisier, as expected:
[ 158 ]
Image Enhancement Chapter 4
Smoothing by averaging with the box blur kernel
The following code block shows how to use the PIL ImageFilter.Kernel() function and
box blur kernels (mean filters) of a size of 3 x 3 and 5 x 5 to smooth a noisy image:
im = Image.open('../images/mandrill_spnoise_0.1.jpg')
pylab.figure(figsize=(20,7))
pylab.subplot(1,3,1), pylab.imshow(im), pylab.title('Original Image',
size=30), pylab.axis('off')
for n in [3,5]:
box_blur_kernel = np.reshape(np.ones(n*n),(n,n)) / (n*n)
im1 = im.filter(ImageFilter.Kernel((n,n), box_blur_kernel.flatten()))
pylab.subplot(1,3,(2 if n==3 else 3))
plot_image(im1, 'Blurred with kernel size = ' + str(n) + 'x' + str(n))
pylab.suptitle('PIL Mean Filter (Box Blur) with different Kernel size',
size=30)
pylab.show()
The following screenshot shows the output of the previous code. As can be seen, the output
image is obtained by convolving the larger size box-blur kernel with the noisy image
smoothed:
[ 159 ]
Image Enhancement Chapter 4
Smoothing with the Gaussian blur filter
The Gaussian blur filter is also a linear filter but, unlike the simple mean filter, it takes the
weighted average of the pixels inside the kernel window to smooth a pixel (the weight
corresponding to a neighbor pixel decreases exponentially with the distance of the neighbor
from the pixel). The following code shows how the PIL ImageFilter.GaussianBlur()
can be used to smooth a noisier image with different radius parameter values for the kernel:
im = Image.open('../images/mandrill_spnoise_0.2.jpg')
pylab.figure(figsize=(20,6))
i = 1
for radius in range(1, 4):
im1 = im.filter(ImageFilter.GaussianBlur(radius))
pylab.subplot(1,3,i), plot_image(im1, 'radius = ' +
str(round(radius,2)))
i += 1
pylab.suptitle('PIL Gaussian Blur with different Radius', size=20)
pylab.show()
[ 160 ]
Image Enhancement Chapter 4
The following screenshot shows the output. As can be seen, with a higher radius with the
Gaussian filter, the image becomes smoother with the removal of more and more noise,
while at the same time blurring the image more:
Comparing smoothing with box and Gaussian
kernels using SciPy ndimage
We can apply a linear filter to smooth images using SciPy's ndimage module functions too.
The following code snippet shows a demonstration of the results of applying the linear
filters on the mandrill image degraded with impulse (salt-and-pepper) noise:
from scipy import misc, ndimage
import matplotlib.pylab as pylab
im = misc.imread('../images/mandrill_spnoise_0.1.jpg')
k = 7 # 7x7 kernel
im_box = ndimage.uniform_filter(im, size=(k,k,1))
s = 2 # sigma value
t = (((k - 1)/2)-0.5)/s # truncate parameter value for a kxk gaussian
kernel with sigma s
im_gaussian = ndimage.gaussian_filter(im, sigma=(s,s,0), truncate=t)
fig = pylab.figure(figsize=(30,10))
pylab.subplot(131), plot_image(im, 'original image')
pylab.subplot(132), plot_image(im_box, 'with the box filter')
pylab.subplot(133), plot_image(im_gaussian, 'with the gaussian filter')
pylab.show()
[ 161 ]
Image Enhancement Chapter 4
The following screenshot shows the output of the previous code. As can be seen, the box
filter of the same kernel size blurs the output image more than the Gaussian filter of same
size with σ=2:
Nonlinear noise smoothing
Nonlinear (spatial) filters also operate on neighborhoods and are implemented by sliding a
kernel (mask) over an image like a linear filter. However, the filtering operation is based
conditionally on the values of the pixels in the neighborhood, and they do not explicitly use
coefficients in the sum-of-products manner in general. For example, noise reduction can be
effectively done with a non-linear filter whose basic function is to compute the median
gray-level value in the neighborhood where the filter is located. This filter is a nonlinear
filter, since the median computation is a non-linear operation. Median filters are quite
popular since, for certain types of random noise (for example, impulse noise), they provide
excellent noise-reduction capabilities, with considerably less blurring than linear smoothing
filters of similar size. Non-linear filters are more powerful than linear filters; for example, in
terms of suppression of non-Gaussian noise such as spikes and for edge/texture preserving
properties. Some examples of non-linear filters are median, bilateral, non-local means, and
morphological filters. The following sections demonstrate the implementation of a few non-
linear filters with PIL, scikit-image, and scipy ndimage library functions.
[ 162 ]
Image Enhancement Chapter 4
Smoothing with PIL
The PIL ImageFilter module provides a set of functions for non-linear denoising of an
image. In this section, we shall demonstrate some of them with examples.
Using the median filter
The median filter replaces each pixel with the median of the values of its neighbor pixels.
This filter is great for removing of salt-and-pepper noise, although it removes small details
from the image. We need to give first rank to the neighborhood intensities and then select
the middle value. Median filtering is resilient to statistical outliers, incurs less blurring, and
is simple to implement. The following code block shows how the PIL ImageFilter
module's MedianFilter() function can be used to remove salt-and-pepper noise from the
noisy mandrill image, with different levels of noise added and a different size of the kernel
window used for the median filter:
i = 1
pylab.figure(figsize=(25,35))
for prop_noise in np.linspace(0.05,0.3,3):
im = Image.open('../images/mandrill.jpg')
# choose 5000 random locations inside image
n = int(im.width * im.height * prop_noise)
x, y = np.random.randint(0, im.width, n), np.random.randint(0,
im.height, n)
for (x,y) in zip(x,y):
im.putpixel((x, y), ((0,0,0) if np.random.rand() < 0.5 else
(255,255,255))) # geenrate salt-and-pepper noise
im.save('../images/mandrill_spnoise_' + str(prop_noise) + '.jpg')
pylab.subplot(6,4,i)
plot_image(im, 'Original Image with ' + str(int(100*prop_noise)) +
'%added noise')
i += 1
for sz in [3,7,11]:
im1 = im.filter(ImageFilter.MedianFilter(size=sz))
pylab.subplot(6,4,i), plot_image(im1, 'Output (Median Filter size='
+ str(sz) + ')')
i += 1
pylab.show()
[ 163 ]
Image Enhancement Chapter 4
The following screenshot shows the output of the previous code, the output images after
applying the median filter on the noisy images with different levels of added noise with a
different kernel size. As can be seen, the results show that the non-linear median filter
works much better for impulse (salt-and-pepper) noise than the linear mean and weighted
mean (Gaussian) filters, although with some patchy effects and the loss of some details:
[ 164 ]
Image Enhancement Chapter 4
Using max and min filter
The following code shows how to use the MaxFilter()to remove the pepper noise
followed by a MinFilter() to remove the salt noise from an image:
im = Image.open('../images/mandrill_spnoise_0.1.jpg')
pylab.subplot(1,3,1)
plot_image(im, 'Original Image with 10% added noise')
im1 = im.filter(ImageFilter.MaxFilter(size=sz))
pylab.subplot(1,3,2), plot_image(im1, 'Output (Max Filter size=' + str(sz)
+ ')')
im1 = im1.filter(ImageFilter.MinFilter(size=sz))
pylab.subplot(1,3,3), plot_image(im1, 'Output (Min Filter size=' + str(sz)
+ ')', size=15)
pylab.show()
The following screenshot shows the output of the previous code block. It can be seen that
the maximum and minimum filters are somewhat effective in removing the salt-and-
pepper noise from the noisy image, respectively:
Smoothing (denoising) with scikit-image
The scikit-image library also provides a set of non-linear filters in the restoration
module. In the following sections, we will discuss about a couple of such very useful filters,
namely bilateral and non-local means filters.
[ 165 ]
Image Enhancement Chapter 4
Using the bilateral filter
The bilateral filter is an edge-preserving smoothing filter. For this filter, the center pixel is
set to the weighted average of the pixel values of some of its neighbors only the ones with
roughly similar brightness as the center pixel. In this section, we shall see how we can use
scikit-image package's bilateral filter implementation to denoise an image. Let us first
start by creating a noisy image from the following gray scale mountain image:
The following code block demonstrates how to use the numpy random_noise()function:
im = color.rgb2gray(img_as_float(io.imread('../images/mountain.png')))
sigma = 0.155
noisy = random_noise(im, var=sigma**2)
pylab.imshow(noisy)
[ 166 ]
Image Enhancement Chapter 4
The following screenshot shows the noisy image created by adding random noise with the
original image using the previous code:
The following code block demonstrates how to use the bilateral filter to denoise the
previous noisy image, with different values for the parameters, σcolor and σspatial:
pylab.figure(figsize=(20,15))
i = 1
for sigma_sp in [5, 10, 20]:
for sigma_col in [0.1, 0.25, 5]:
pylab.subplot(3,3,i)
pylab.imshow(denoise_bilateral(noisy, sigma_color=sigma_col,
sigma_spatial=sigma_sp, multichannel=False))
pylab.title(r'$\sigma_r=$' + str(sigma_col) + r', $\sigma_s=$' +
str(sigma_sp), size=20)
i += 1
pylab.show()
The following screenshot shows the output of the previous code. As can be seen, if the
standard deviation is higher, the image gets less noisy but more blurred. It takes a few
minutes to execute the previous code block, as the implementation is even slower on RGB
images:
[ 167 ]
Image Enhancement Chapter 4
Using non-local means
Non-local means is a non-linear denoising algorithm that preserves textures. In this
algorithm, for any given pixel, a weighted average of values of only those nearby pixels
that have similar local neighbors as the pixel of interest are used to set the value of the
given pixel. In other words, small patches centered on the other pixels are compared to the
patch centered on the pixel of interest. In this section, we demonstrate the algorithm by
denoising a noisy parrot image using the non-local means filter. The h parameter to the
function controls the decay in patch weights as a function of the distance between patches.
If h is large, it allows more smoothing between dissimilar patches. The following code block
shows how to denoise with non-local means:
def plot_image_axes(image, axes, title):
axes.imshow(image)
axes.axis('off')
axes.set_title(title, size=20)
[ 168 ]
Image Enhancement Chapter 4
parrot = img_as_float(imread('../images/parrot.png'))
sigma = 0.25
noisy = parrot + sigma * np.random.standard_normal(parrot.shape)
noisy = np.clip(noisy, 0, 1)
# estimate the noise standard deviation from the noisy image
sigma_est = np.mean(estimate_sigma(noisy, multichannel=True))
print("estimated noise standard deviation = {}".format(sigma_est))
# estimated noise standard deviation = 0.22048519002358943
patch_kw = dict(patch_size=5, # 5x5 patches
patch_distance=6, # 13x13 search area
multichannel=True)
# slow algorithm
denoise = denoise_nl_means(noisy, h=1.15 * sigma_est, fast_mode=False,
**patch_kw)
# fast algorithm
denoise_fast = denoise_nl_means(noisy, h=0.8 * sigma_est, fast_mode=True,
**patch_kw)
fig, axes = pylab.subplots(nrows=2, ncols=2, figsize=(15, 12), sharex=True,
sharey=True)
plot_image_axes(noisy, axes[0, 0], 'noisy')
plot_image_axes(denoise, axes[0, 1], 'non-local means\n(slow)')
plot_image_axes(parrot, axes[1, 0], 'original\n(noise free))
plot_image_axes(denoise_fast, axes[1, 1], 'non-local means\n(fast)')
fig.tight_layout()
# PSNR metric values
psnr_noisy = compare_psnr(parrot, noisy)
psnr = compare_psnr(parrot, denoise.astype(np.float64))
psnr_fast = compare_psnr(parrot, denoise_fast.astype(np.float64))
print("PSNR (noisy) = {:0.2f}".format(psnr_noisy))
print("PSNR (slow) = {:0.2f}".format(psnr))
print("PSNR (fast) = {:0.2f}".format(psnr_fast))
# PSNR (noisy) = 13.04 # PSNR (slow) = 26.25 # PSNR (fast) = 25.84
pylab.show()
The following screenshot shows the output. As can be seen, the slow version of the
algorithm achieves better PSNR than the faster version, representing a trade-off. Both of the
algorithm output images have much higher PSNR than the noisy image:
[ 169 ]
Image Enhancement Chapter 4
Smoothing with scipy ndimage
The scipy ndimage module provides a function named percentile_filter(), which is a
generic version of the median filter. The following code block demonstrates how to use this
filter:
lena = misc.imread('../images/lena.jpg')
# add salt-and-pepper noise to the input image
noise = np.random.random(lena.shape)
lena[noise > 0.9] = 255
lena[noise < 0.1] = 0
plot_image(lena, 'noisy image')
pylab.show()
fig = pylab.figure(figsize=(20,15))
i = 1
for p in range(25, 100, 25):
for k in range(5, 25, 5):
pylab.subplot(3,4,i)
filtered = ndimage.percentile_filter(lena, percentile=p,
size=(k,k,1))
plot_image(filtered, str(p) + ' percentile, ' + str(k) + 'x' +
str(k) + ' kernel')
i += 1
pylab.show()
[ 170 ]
Image Enhancement Chapter 4
The following screenshot shows the output of the previous code. As can be seen, out of all
of the percentile filters, the median filter (corresponding to the 50th percentile) with a small
kernel size does the best to remove salt-and-pepper noise, while at the same time losing the
fewest possible details in the image:
Summary
In this chapter, we discussed different image enhancement methods, starting from point
transformations (for example, contrast stretching and thresholding), then techniques based
on histogram processing (for example, histogram equalization and histogram matching),
followed by image denoising techniques with linear (for example, mean and Gaussian) and
non-linear (for example, median, bilateral, and non-local means) filters.
[ 171 ]
Image Enhancement Chapter 4
By the end of this chapter, the reader should be able to write Python codes for point
transformations (for example, negative, power-law transform, and contrast stretching),
histogram-based image enhancements (for example, histogram equalization/matching), and
image denoising (for example, mean/median filters).
In the following chapter, we shall continue discussing more image enhancement techniques
based on image derivatives and gradients.
Questions
1. Implement histogram matching for colored RGB images.
2. Use the equalize() function from skimage.filters.rank to implement local
histogram equalization and compare it with the global histogram equalization
from skimage.exposure with a grayscale image.
3. Implement Floyd-Steinberg error-diffusion dithering using the algorithm
described here https://en.wikipedia.org/wiki/Floyd%E2%80%93Steinberg_
dithering and convert a grayscale image into a binary image.
4. Use ModeFilter() from PIL for linear smoothing with an image. When is it
useful?
5. Show an image that can be recovered from a few noisy images obtained by
adding random Gaussian noise to the original image by simply taking the
average of the noisy images. Does taking the median also work?
Further reading
http://paulbourke.net/miscellaneous/equalisation/
https://pdfs.semanticscholar.org/presentation/3fb7/
fa0fca1bab83d523d882e98efa0f5769ec64.pdf
https://www.comp.nus.edu.sg/~cs4243/doc/SciPy%20reference.pdf
https://en.wikipedia.org/wiki/Floyd%E2%80%93Steinberg_dithering
https://en.wikipedia.org/wiki/Floyd%E2%80%93Steinberg_dithering
[ 172 ]
