L14: Optimal linear filtering
– Wiener filtering
Lennart Svensson
Department of Signals and Systems
Chalmers University of Technology
Problem formulation
General solution
Filtering solutions
Reviewing L11-L13
What have we done so far?
Signal models (L11-L12)
Nonparametric models: ACF and PSD.
Parametric models: AR, MA and ARMA.
Signal model estimation (L13)
Nonparametric spectral estimation: the periodogram.
Pros:
fast to compute
asymptotically unbiased.
Cons:
limited resolution for finite N:
the modified periodogram improves this
large variance for all N:
Blackman-Tukey’s method lowers variance.
Parametric spectral estimation: AR-estimation.
1 Estimate rx [k] from data.
2 Reformulate Yule-Walker to get â = Rx−1 rx .
Chalmers University of Technology Lennart Svensson 2/12
Problem formulation
General solution
Filtering solutions
Learning objectives
After today’s lecture you should be able to
explain what type of problems Wiener-filters can solve.
derive the Wiener-Hopf (WH) equations.
use the WH-equations to derive a causal FIR Wiener filter.
use the WH-equations to derive a non-causal IIR Wiener
filter.
Compute the mean squared error (MSE) of a Wiener-filter.
Chalmers University of Technology Lennart Svensson 3/12
Problem formulation
General solution Filtering, smoothing and prediction
Filtering solutions
Filtering, smoothing and prediction
Let s[n] and w [n] be zero mean, wide sense stationary
processes and
x[n] = s[n] + w [n].
Objective
Select H(z) to make e[n] as ”small” as possible
d̂ [n]
x[n] H(z) e[n] = d̂ [n] − d [n]
−
d [n]
Small could mean different things. We use mean squared error
E e[n]2 ,
since this is easy to minimize.
Chalmers University of Technology Lennart Svensson 4/12
Problem formulation
General solution Filtering, smoothing and prediction
Filtering solutions
Filtering, smoothing and prediction
Based on measurements collected up until now, we encounter three
common problems (k > 0):
Filtering – estimating current signal values, d [n] = s[n].
Applications: positioning, control systems, noise or echo
cancellation, etc.
Smoothing – estimating past signal values, d [n] = s[n − k].
Applications: signal analysis, image processing, system
identification (modelling).
Prediction – estimating future signal values, d [n] = s[n + k].
Applications: decision making, planning, weather forecasts,
etc.
Chalmers University of Technology Lennart Svensson 5/12
Problem formulation
General solution Filtering, smoothing and prediction
Filtering solutions
Filtering, smoothing and prediction
These problems can be illustrated as
time
Prediction:
Filtering:
Smoothing:
Time of interest
Measurements
Chalmers University of Technology Lennart Svensson 6/12
Problem formulation
General solution Filtering, smoothing and prediction
Filtering solutions
Filtering, smoothing and prediction
We seek a linear estimator (filter)
X
d̂ [n] = h[n] ? x[n] = h[k]x[n − k]
k
of d [n].
As mentioned above, we wish to minimize the mean square
error (MSE),
!2
X
MSE(h) = E d [n] − h[k]x[n − k]
k
where the vector h contains the impulse response coefficients
h[k].
The resulting Wiener filter d̂ [n] is a linear minimum mean
square error (LMMSE) estimator.
Chalmers University of Technology Lennart Svensson 7/12
Problem formulation
General solution Wiener-Hopf equations
Filtering solutions
Wiener-Hopf (W-H) equations
The W-H equations are very important and can be used to
solve all the problems mentioned above.
Objective: (again) We wish to minimize
!2
X
MSE(h) = E d [n] − h[k]x[n − k]
k
with respect to h.
Derivation 1: the function is quadratic in h
⇒ it is convex in h
⇒ no local optima (except for the global optimum)
⇒ sufficient to differentiate and set to zero!
Chalmers University of Technology Lennart Svensson 8/12
Problem formulation
General solution Wiener-Hopf equations
Filtering solutions
Wiener-Hopf (W-H) equations
Differentiate the MSE:
!2
∂ ∂ X
MSE(h) = E d [n] − h[k]x[n − k]
∂h[t] ∂h[t]
k
( ! )
X
= E 2 d [n] − h[k]x[n − k] (−x[n − t])
k
X
= −2rdx [t] + 2 h[k]rx [t − k]
k
Setting this derivative to zero gives the
Wiener-Hopf (WH) equations
X
h[k]rx [t − k] = rdx [t],
k
for all t where h[t] is free to select.
Chalmers University of Technology Lennart Svensson 9/12
Problem formulation
Causal FIR filters
General solution
MSE of optimal FIR filter
Filtering solutions
FIR filters
Suppose H(z) is a causal FIR filter:
p−1
X
d̂ [n] = h[k]x[n − k].
n=0
The W-H eq’s can be written as
rx [0] rx [1] ... rx [p − 1] h[0] rdx [0]
rx [1] r x [0] ... rx [p − 2] h[1] rdx [1]
.. .. .. .. = ..
..
. . . . . .
rx [p − 1] rx |p − 2] ... rx [0] h[p − 1] rdx [p − 1]
| {z } | {z } | {z }
Rx h rdx
which yields that
hopt = R−1
x rdx .
Chalmers University of Technology Lennart Svensson 10/12
Problem formulation
Causal FIR filters
General solution
MSE of optimal FIR filter
Filtering solutions
What is the minimum MSE?
The minimum MSE can be calculated by plugging in hopt :
2 n o n o
E emin [n] = E emin [n] d [n] − d̂opt [n] = Note: d̂opt ⊥ e[n]
p−1
( ! )
X
=E d [n] − hopt [k]x[n − k] d [n]
k=0
p−1
X
T −1
= rd [0] − hopt [k]rdx [k] = rd [0] − rdx Rx rdx
k=0
Special case: if d [n] and x[n] are uncorrelated, then d̂ [n] = 0
and the MSE is rd [0].
In general, the more correlated (similar) x[n] is to d [n] the
better is the estimate d̂ [n]!
Chalmers University of Technology Lennart Svensson 11/12
Problem formulation
Causal FIR filters
General solution
MSE of optimal FIR filter
Filtering solutions
Learning objectives
After today’s lecture you should be able to
explain what type of problems Wiener-filters can solve.
derive the Wiener-Hopf (WH) equations.
use the WH-equations to derive a causal FIR Wiener filter.
use the WH-equations to derive a non-causal IIR Wiener
filter.
Compute the mean square error (MSE) of a Wiener-filter.
Chalmers University of Technology Lennart Svensson 12/12
