Image Restoration

Noise Removal Using

Frequency Domain Filters
Image restoration
• Image restoration is the process of reconstructing or
improving an image that has been corrupted or degraded.

• Unlike image enhancement, which improves the visual appeal,

restoration focuses on reversing the damage by modeling the
degradation and applying corrective techniques.

• The ultimate goal is to restore the image to its original form or

produce the most accurate representation of the original scene
Image Restoration
 Image enhancement Image Restoration
the original image is processed so that the The aim of image restoration is to bring
resultant image is more suitable than the the image towards what it would have
original for specific applications. been if it had been recorded without
degradation.
Image enhancement makes a picture look Image restoration tries to fix the image to
better, without regard to how it really truly get back to the real, true image.
should look.
Image enhancement: “improve” an image Image restoration: remove distortion from
subjectively image in order to go back to the
“original” ➔ objective process.
Image enhancement is a purely subjective Image restoration is an objective process.
processing technique.
Image enhancement is a cosmetic Restoration tries to reconstruct by using a
procedure i.e. it does not add any extra priori knowledge of the degradation
information to the original image. It phenomena. Restoration hence deals with
merely improves the subjective quality of getting an optimal estimate of the desired
the images by work in with the existing result
data.
image restoration
■ started from the 1950s
■ application domains
■ Scientific explorations
■ Legal investigations
■ Film making and archival
■ Image and video (de-)coding
■ Consumer photography
A model of the image degradation/restoration
process

g(x,y)=f(x,y)*h(x,y)+N(x,y) – Spatial domain

G(u,v)=F(u,v). H(u,v)+N(u,v) – Frequency doma
A model of the image degradation/
restoration process
Where,
f(x,y) -input image
f^(x,y) - estimated original image
g(x,y) - degraded image
h(x,y) -degradation function
N(x,y) - additive noise term
Classification of Restoration
methods
• Deterministic or stochastic methods.
In deterministic methods, we work directly with the image
values in either space or frequency domain.
In stochastic methods, we work with the statistical properties
ofthe image of interest (autocorrelation function,
covariance function, variance, mean etc.)

• Non-blind or semi-blind or blind methods

In non-blind methods the degradation process is known.
In semi-blind methods the degradation process is partly-
known.
In blind methods the degradation process is unknown.
implementation types of restoration
methods
• Direct methods
The signals we are looking for (original “undistorted” image
and degradation model) are finally obtained through a
signle closed-form expression.
• Iterative methods
The signals we are looking for are obtained through a
mathematical procedure that generates a sequence of
improving approximate solutions to the problem.
usual assumptions for the
distortion model
1. Noise
■ Independent of spatial location

■ Exception: periodic noise ,Gaussian, speckle …

■ Uncorrelated with image

2. Degradation function H
■ Linear

■ Position-invariant
1. Noise

Noise tells unwanted information in digital images. •

Noise produces undesirable effects such as artifacts,
unrealistic edges, unseen lines, corners, blurred
objects and disturbs background scenes. To reduce
these undesirable effects.
1. Sources of noise
• Image Restoration refers to a class of methods that aim
at reducing or removing various types of distortions of
the image of interest. These can be:
 Distortion due to sensor noise.
 Out-of-focus camera.
 Motion blur.
 Weather conditions.
 Scratches, holes, cracks caused by aging of the image.
 Image acquisition (digitization) (resolution)
 Image transmission - Interference can be added to an image
during transmission
 Others.
Sources of noise
• Spatial properties of noise
– Statistical behavior of the gray-level values of
pixels
– Noise parameters, correlation with the image
• Frequency properties of noise
– Fourier spectrum
– Ex. white noise (a constant Fourier spectrum)
Motion blur: A typical type of
degradation
Types of motion blur
• 1-D uniform motion blur

• 2-D uniform motion blur

Atmospheric turbulence: A
typical type of degradation
Typical model for atmospheric
turbulence
• Atmospheric turbulence

negligible
  k  0.0025
distortion

k  0.001   k  0.00025
degraded images

ideal image

■ What caused the image to blur?

■ Camera: translation, shake, out-of-focus …
■ Environment: scattered and reflected light
■ Device noise: CCD/CMOS sensor and circuitry
■ Quantization noise
 Uniform out-of-focus blur: A typical type of
degradation
• Uniform out-of-focus blur:
Degradations
• original

• optical blur

• motion blur

• spatial quantization (discrete pixels)

• additive intensity noise

recovering from noise
■ overall process
1. Observe and estimate noise type
2. Determine the type of filter and it’s parameters
3. apply optimal (spatial) filtering.
4. (if rerstorted image is done ) end procedure .
else adjust filter type/parameters and return (2,3,4)
■ Example noise-reduction filters
1. Mean/median filter family
2. Adaptive filter family
3. Other filter family e.g. Homomorphic filter for
multiplicative noise
example: Gaussian noise
example: salt-and-pepper noise
Recovering from Periodic Noise
• periodic noise is generated from electronics interferences,
especially in power signal during image acquisition.
• This noise has special characteristics like spatially
dependent and sinusoidal in nature at multiples of
specific frequency.
• It appears in form of conjugate spots in frequency
domain.
• It can be conveniently removed by using a narrow band
reject filter or notch filter.
X-Ray image corrupted by periodic noise.
a Original image,
b noise pattern,
c noisy image in spatial domain,
d noisy image in frequency domain
periodic Noisesources
1. Nature
2. Electricity network
3. Electronics devices.
Original image Periodic noise
Originalimage | Periodicnoise

(a) (b)
Figures: a) Source image. b) Corrupted image with periodic noise in spatial domain.
 amplitude spectrum and periodic
noise
• Unfortunately, an efficient spatial filter for periodic
noise reduction in an image has not been developed yet.
• However, recovering the
image tends to become
easier in the frequency
domain because of
evident noise peaks
as shown in Figure. Fig. 2D demonstration of peak points in a noisy image's
amplitude spectrum.
periodic Noisefilters
- it will be better to use the frequency domain to
remove noise, so the frequency domain filters are used.
The most used filters:

1.Band-reject Filters

2.Band-pass Filters

3.Notch Filters
Recovering from Periodic Noise
Butterworth LPF Butterworth bandreject filter
Example of bandreject filter
Band-pass Filter
• Bandpass filtering is usually used to isolate
components of an image that correspond to a
band of frequencies can also be used to isolate
noise interference.

𝐻𝐵𝑃 𝑢, 𝑣 = 1 − 𝐻𝐵𝑅(𝑢, 𝑣)
 NotchFilter
Notch-filters help to eliminate noises from digital images.
It is a kind of band-reject/band- pass filter that rejects/passes a
very narrow set of frequencies, around a center frequency.

𝐻𝑁𝑃 𝑢, 𝑣 = 1 − 𝐻𝑁𝑅(𝑢, 𝑣)
Notch Filter

Original Image
Notch Filter

Image corrupted with

sinusoidal noise
Notch Filter

Restored Image
notch filter
2. Properties of the degradation function

Properties of the degradation function H (u,v)

•Linear system

– H[af1(x,y)+bf2(x,y)]=aH[f1(x,y)]+bH[f2(x,y)]

•Position(Time)-invariant system
– H[f(x,y)] is time invariant if
H[f(x, y)]=H[f(x- n0, y-n0)]
Advantage and drawback of LSI
• Advantage

It is much easier to deal with linear and space-invariant models.

 Mathematics are easier.

 The distorted image is the convolution of the original image

and the distortion model.

 Software tools are available.

• Disadvantage

For various realistic types of image degradations assumptions for

linearity and space-invariance are too strict and significantly
deviate from the true degradation model.
Estimation of Degradation Function
Degradation model:
g( x, y) =f( x, y) h( x, y)+noise (x,y)
or
G (u, v) = F (u, v) H (u, v) + Noise (u, v)
Methods:
1.Estimation by Image Observation
2.Estimation by Experiment
3.Estimation by Modeling
 Estimation degradation function H(u,v) by PSF
 Inverse filter.
 The Wiener filter.
1. Estimation by Image Observation
f(x,y)*h(x,y) g(x,y)

F(x,y)
Original image (unknown)
Degraded image Observation
DFT Subimage
Estimated Transfer
function

Restoration
process by
estimation
DFT
Reconstructed
This case is used when: Subimage

•we know only g(x,y) and

•cannot repeat the experiment!
2. Estimation by Experiment
1-D impulse response
Response image from
Input impulse image the system

System
H()

 (x,y) g( x, y)
DFT DFT

DFT [(x,y)] A G (u, v)

G (u, v)
H (u, v) 
A
2. Estimation by Experiment
If the system is linear and time-invariant, the image
formed by the system is the convolution of the object with
the PSF:

where:
• I(x,y) is the observed image,
•g(x,y) is the degradation image,
•PSF(x,y)is the point spread
function,
•∗ denotes convolution
Multi-D impulse response
•The point spread function (PSF): is the image of an
idealized point source (infinitely small and bright) after it
passes through an optical system.
•It represents the system's response
to a point input, accounting for
diffraction, aberrations,
and other imperfections.

PSF formula
Point spread function ( PSF)
• Most types of degradation function that can be analyzed are
some form of blurring (a point being spread out).

• This type of degradation function is also referred to as point

spread function (PSF).

• PSF (point spread function) is the spatial-domain impulse

response of the optical imaging system assuming it's
modeled as an LTI system.
Point spread function ( PSF) types
• PSF actually measures the amount and direction of blurring
that can occur to one pixel.

• The blur can be:

• Linear in one direction (horizontal, vertical, diagonal).
• Circularly symmetric (blur in all directions).

• This blurring is modeled using a PSF mask.

• The PSF mask is then used to model h(r,c).

Blur can be linear in one direction
only horizontal ,verttical,,,,,

0 0 0 0 0 0 0 1 0 0 1 0 0 0 0
0 0 0 0 0 0 0 1 0 0 0 2 0 0 0
1 1 1 1 1 0 0 4 0 0 0 0 4 0 0
0 0 0 0 0 0 0 1 0 0 0 0 0 2 0
0 0 0 0 0 0 0 1 0 0 0 0 0 0 1
Horizontal PSF mask Vertical PSF mask with Diagonal PSF mask
with uniform blur center-weighting with Gaussian distribution
Blur can be Circularly symmetric (blur
in all directions)
0 1 1 1 0
1 2 4 2 1
1 4 8 4 1
1 2 4 2 1
0 1 1 1 0
Circular PSF mask with Gaussian distribution
 Point Spread Function

PSF

Send a Point of light through the system and see

what comes out.
PSF algorithm :
•Send a point of light through the system.
• The frequency domain filters incorporate information
regarding the noise and the PSF into their model.
• G(u,v) = H(u,v)I(u,v) + N(u,v)

• To obtain the restored image, the general form is as follows:

• Î(r,c) = F -1[Î(u,v)] = F -1[R (u,v). G(u,v)]
• R (u,v) is the frequency domain PSF.
3. Estimation by Modeling
used when we know physical mechanism underlying the
image formation process that can be expressed
mathematically.
Original image severe image

k = 0.0025 Example: Atmospheric

Turbulence model

 ek( u v )
2 2 5 /6
H (u, v)
Mild turbulence Low turbulence

k = 0.001 k = 0.00025
Estimation by Modeling: Motion Blurring
Assume that camera velocity is ( x0(t), y0(t))The
blurred image is obtained by
Estimation by Modeling: Motion Blurring

Then we get, the motion blurring transfer function:

Motion Blurring Example

Motion blurred image

Original image
a =b = 0.1,T = 1
Inverse Filter
• A technique used in signal processing and image
processing to recover an original signal or image from a
blurred noise.

• Inverse filtering is a deterministic and direct method for

image restoration .

• The inverse filter is the term 1/H(u,v).

• Note that the inversion is a point-by-point inversion
and not a matrix inversion.
Inverse Filter
• To find 1/H(u,v), we take each term separately and
divide it into 1.
50 50 25
H ( u , v )   20 20 20
 

 20 35 22 

 1 1 1 
 50 50 25 
1  1 1 1 
  
H ( u, v )  20 20 20 
 1 1 1 

 20 35 22 
inverse filter
■ assume h is known: low-pass filter H(u,v)

■ inverse filter
■ recovered image
H(u,v)
inverse filter
• The degraded image G(u,v) is :

G(u,v) = H(u,v)I(u,v) + N(u,v)

• If N(u,v) =0 ( no noise in the system )

• G(u,v) = H(u,v)I(u,v)

• ෠ ෡ 𝑢, 𝑣
𝐹 𝑢, 𝑣 = 𝐺 𝑢, 𝑣 . 𝐻
1
𝐹෠ 𝑢,𝑣 = 𝐻 u,v I u,v . ෡ = 𝐼(𝑢,𝑣)
𝐻 𝑢,𝑣
inverse filtering example

loss of
information
Inverse filter(noisy scenarios)
Inverse filter(noisy scenarios)
• inverse filtering with cutoff (lowpass) to suppress noise
• Note that the most popular types of degradations are low
pass filters (out- of-focus blur, motion blur).
Inverse filter(noisy scenarios)
Inverse Filter
• One method to deal with this problem is to limit the
restoration to a specific radius about the origin in the
spectrum, called the restoration cutoff frequency.

• This is equivalent to an ideal lowpass filter, which results

in blurring and ringing (waves around the edges).
Inverse Filter
• Setting the cut-off frequency too low may provide poor
result (blurred).

• Setting it too high may result in the restored image being

overshadowed by noise.

• In practice, the cut-off frequency must be experimentally

determined and highly application specific.
Inverse Filter

Original Image Image blurred with 11x11

Gaussian convolution mask
Inverse Filter

Inverse filter, cutoff frequency = 32 Inverse filter, cutoff frequency = 64

Histogram stretched to show details Histogram stretched to show details
 Inverse Filter

Inverse filter, cutoff frequency = 95 Inverse filter, cutoff frequency = 120

Histogram stretched to show details Histogram stretched to show details
Wiener Filter
• The most important technique for removal of blur in images
due to linear motion or unfocussed optics is the Wiener filter
• The Wiener filtering executes an optimal tradeoff between
inverse filtering and noise smoothing.
• It removes the additive noise and inverts the blurring
simultaneously.
• It works on the assumption that additive noise is a stationary
random process, independent of pixel location(uncorrolated
pixel between image and noise).
 stationary random process
A random process at a given time is a random variable and, in
general, the characteristics of this random variable depend on
the time at which the random process is sampled. A random
process X(t) is said to be stationary or strict-sense
stationary if the pdf (probability density function) of any set
of samples does not vary with time.
How does the Wiener filter work?
• the algorithm minimizes the square error between the
original and reconstructed images.

• It can solve some of the weaknesses of the inverse filter.

• This is done by attempting to model the error in the
restored image using statistical methods.
• The model assumes that the image and the noise are
random processes that are uncorrelated and have known
power spectra.
How does the Wiener filter work?
How does the Wiener filter work?
How does the Wiener filter work?
Wiener Filter
• The Wiener filter is defined as follows:

H * (u , v)
RW (u , v) 
S n (u , v)
H (u , v) 
2

S I (u , v)
H*(u,v) = complex conjugate of H(u,v)
Sn(u,v) = |N(u,v)|2 = power spectrum of the noise
SI (u,v) = |I(u,v)|2= power spectrum of the original image
Wiener Filter (free noise)
• If the noise power spectrum is zero, the Wiener filter
becomes an inverse filter.
H * (u, v)
RW (u, v) 
0
H (u, v) 
2

S I (u, v)
H * (u, v) H * (u, v) 1
 2
 *

H (u, v) H (u, v) H (u, v) H (u, v)
Wiener Filter
• When the noise spectrum gets larger, the denominators of
Wiener filter increases, thus decreasing the value of
W(u,v).

• This leads to a desired behavior:

• In portions of the spectrum heavily corrupted by noise,
the filter attenuates the signal.
Wiener Filter
• In practical applications the original, uncorrupted
image is typically not available.
• Therefore, the power spectrum ratio is replaced by a
parameter K whose optimal value must be
experimentally determined.
*
H (u , v)
RW (u , v) 
H (u , v)  K
2
Wiener Filter
• K can just be a constant.

• However, making K a function of the frequency domain

variables (u,v) may give some extra benefits.
• Noise typically dominates at high frequencies.
• Making the value K increases as the frequency increases
will cause the filter to attenuates at high frequency, thus
reducing the noise.
Schematic effect of Wiener filter
Wiener Filter example
H* (u, v)
W(u, v) 
H(u, v)  K
2
Wiener filter example

Wiener filter is more robust to noise, and preserves high-

frequency details.
Wiener filter example

Ringing effect visible, too many high frequency

components?

(a) Blurry image (b) restored with wiener filter

(c) restored with inverse
Wiener Filter

Original Image Blurred image with added

gaussian noise
Wiener Filter

Result of Wiener filter Result of inverse filter

with cut-off frequency=80 with cut-off frequency=80
 degraded inverse-filtered Wiener-filtered

motion blur
+ noise

noise*10-1

noise*10-5
Another example: reading licence plates
 advantages of wiener filter disadvantages of wiener filter

One of the main advantages of using a using a wiener filter is that it requires a
wiener filter for noise reduction in DSP is priori knowledge of the power spectra of
that it is optimal in the mean square error the input signal, the noise signal, and the
sense, which means that it minimizes the true signal. This can be difficult or
expected value of the squared error signal. impractical to obtain in many cases ,
especially when the signal and noise are
non-stationary or non-Gaussian.
using a wiener filter is that it is adaptive, a wiener filter is linear, which means that it
which means that it can adjust to the cannot handle nonlinear phenomena such
changing characteristics of the input signal as harmonics, intermodulation, or clipping
and the noise signal.
This can make it suitable for applications wiener filter can introduce some
where the signal and noise statistics are undesirable effects such as ringing,
unknown or vary over time, such as speech overshooting, or blurring, due to its
enhancement, image restoration, or radar frequency-domain smoothing.
detection
Wiener filter: when does it not work?
How much de-blurring is just enough?
Variations of Wiener filters:
Improve Wiener Filter
■ Constrained Least Squares
■ geometric mean filters
■ Blind deconvolution
Constrained Least-Squares Filter
• This filter can eliminate some of the artifacts caused by
other frequency domain filters.
• Some of the other frequency domain filters cause
“waves” to appear in the image.
• This filter includes a smoothing criterion in the filter
derivative that can eliminate this effect.

• The constrained least-squares filter is defined as follows:

Constrained Least-Squares Filter
*
H (u , v)
RCLS (u , v) 
H (u , v)   P(u , v)
2 2

•  = the adjustment factor which is experimentally determined and is

application dependent
• P(u,v) = the Fourier Transform of the smoothness criterion function.

• A standard function used for p(r,c) is the laplacian filter

mask:  0 1 0 
 1 4  1
 
 0  1 0 
Constrained Least-Squares Filter
• However, before P(u,v) is calculated, the p(r,c) function
must be zero-padded to the same size as the image.

• P(u,v) is then obtained by applying Fourier transform on

p(r,c).

• Once P(u,v) is obtained, then it can be incorporated into

RCLS(u,v).
Constrained Least-Squares
Filter

Original Image Blurred image with added

gaussian noise
Constrained Least-Squares Filter

Result of CLS filter with Result of Wiener filter

cut-off frequency = 128 with cut-off frequency=80
 = 23000
Geometric Mean Filters
• The geometric filter equation provides a general
form for many of the frequency domain restoration
filters.
1
 
 H (u, v) 

 * 
RGM (u, v)     H (u, v) 
2
 H (u, v)    S n (u, v)  
 H (u, v)   
2
 
  S I (u, v)  
Geometric Mean Filters
• The terms are as previously defined, with γ and α being
positive real constants.

• If α = ½ and γ = 1, then this filter is called a power

spectrum equalization filter (homomorphic filter).
• It is an average between the inverse filter and the
Wiener filter (hence the term geometric mean).

• If α = 0, this filter is called a parametric Wiener filter.

• α = 1, this filter is called a parametric Inverse filter.

Homomorphic filtering (HF)
• Homomorphic filtering (HF) is a methodology that
separates an image into two components:
 illumination
reflectance.

• The illumination-reflectance model of image formation

says that the intensity at any pixel, which is the amount of
light reflected by a point on the object, is the product of the
illumination of the scene and the reflectance of the object(s)
in the scene, i.e.,
Homomorphic Filtering
𝐼𝑛𝑡𝑒𝑛𝑠𝑖𝑡𝑦 𝑥, 𝑦 = 𝑖 𝑥, 𝑦 . 𝑟 𝑥, 𝑦
𝐼𝑛𝑡𝑒𝑛𝑠𝑖𝑡𝑦𝑙𝑛 = 𝐿𝑛 𝐼𝑛𝑡𝑒𝑛𝑠𝑖𝑡𝑦
𝐼𝑛𝑡𝑒𝑛𝑠𝑖𝑡𝑦𝑙𝑛 = 𝐿𝑛 𝑖 𝑥, 𝑦 + ln[𝑟 𝑥, 𝑦 ]

■ Slow-changing illumination i(x,y)

■ fast- changing reflectance r(x,y)

Homomorphic Filtering
• Then we use a high-pass filter in the log domain to
remove the low-frequency illumination component while
preserving the high-frequency reflectance component.
• The HF is Used to remove multiplicative noise, or
illumination variations
• Also used in to separate excitation and filtering effects in
speech, e.g. hearing aid
Geometric Mean Filters
• The inverse filter is inadequate with too much noise.

• The Wiener filter has the tendency to cause undesirable

artifacts in the resultant image.

• The constrained least-squares filter helps to minimize the

Wiener-type artifact.

• The parametric Wiener and geometric mean provide

additional parameters for application-specific needs.
Blind deconvolution
• The methods for estimating H(u,v) are known as Blind
Deconvolution.

• Wiener filter assumes both the image and noise spectrum

are know (or can be easily estimated), in practice this
becomes trial-and-error since noise and signal parameters
are often hard to obtain.
• our degradation is modeled as a convolution plus noise.

• g(x,y)=f(x,y)*h(x,y)+noise (x,y)

• In the frequency domain, convolution becomes multiplication.

• G(u,v)=H(u,v). F(u,v)+Noise(U,v)

• If we ignore the additive noise, we can take the log of the

multiplication and get addition.

• Log[ G(u,v)= Log[H(u,v)]+ Log[F(u,v)]

• Log[H(u,v)]= Log[ G(u,v)]-Log[F(u,v)]

• The problem with this method is that in practice we
can't ignore the noise. Therefore,

• H is then used with the spectra of original and noise

to compute the Wiener Filter.

• This method only computes Magnitude of H, so its

best for phaseless LSI filters.
The end