MIT OpenCourseWare

http://ocw.mit.edu
2.161 Signal Processing: Continuous and Discrete
Fall 2008
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
e
n
f
n
y
n -

MASSACHUSETTS INSTITUTE OF TECHNOLOGY
DEPARTMENT OF MECHANICAL ENGINEERING
2.161 Signal Processing - Continuous and Discrete
Introduction to Least-Squares Adaptive Filters
1
1 Introduction
In this handout we introduce the concepts of adaptive FIR lters, where the coecients are contin-
ually adjusted on a step-by-step basis during the ltering operation. Unlike the static least-squares
lters, which assume stationarity of the input, adaptive lters can track slowly changing statistics
in the input waveform.
The adaptive structure is shown in Fig. 1. The adaptive lter is FIR of length M with coecients
b
k
, k = 0, 1, 2, . . . , M 1. The input stream {f(n)} is passed through the lter to produce the
sequence {y(n)}. At each time-step the lter coecients are updated using an error e(n) = d(n)
y(n) where d(n) is the desired response (usually based of {f(n)}). The lter is not designed to
causal l i near F R fi l ter
H(z)
Adapti ve Least-Squares
Al gori thm
d
n
+
error
fi l ter coeffi ci ents
Figure 1: The adaptive least-squares lter structure.
handle a particular input. Because it is adaptive, it can adjust to a broadly dened task.
2 The Adaptive LMS Filter Algorithm
2.1 Simplied Derivation
In the length M FIR adaptive lter the coecients b
k
(n), k = 1, 2, . . . , M 1, at time step n are
adjusted continuously to minimize a step-by-step squared-error performance index J(n):

2
M1
J(n) = e
2
(n) = (d(n) y(n))
2
= d(n) b(k)f(n k) (1)
k=0
J(n) is described by a quadratic surface in the b
k
(n), and therefore has a single minimum. At each
iteration we seek to reduce J(n) using the steepest descent optimization method, that is we move
each b
k
(n) an amount proportional to J(n)/b(k). In other words at step n + 1 we modify the
lter coecients from the previous step:
J(n)
b
k
(n + 1) = b
k
(n) (n) , k = 0, 1, 2, . . . M 1 (2)
b
k
(n)
1
D. Rowell December 9, 2008
1
f(n)
k k





where (n) is an empirically chosen parameter that denes the step size, and hence the rate of
convergence. (In many applications (n) = , a constant.) From Eq. (1)
J(n) e
2
(n) e(n)
= = 2e(n) = 2e(n)f(n k)
b
k
b
k
b
k
and the adaptive lter algorithm is
b
k
(n + 1) = b
k
(n) + e(n)f(n k), k = 0, 1, 2, . . . M 1 (3)
or in matrix form
b(n + 1) = b(n) + e(n)f (n), (4)
where
b(n) = [b
0
(n) b
1
(n) b
2
(n) b
M1
]
T
is a column vector of the lter coecients, and
f (n) = [f(n) f(n 1) f(n 2) f(n (M 1))]
T
is a vector of the recent history of the input {f(n)}.
Equation (3), or (4), denes the xed-gain FIR adaptive Least-Mean-Square (LMS) lter algo-
rithm. A Direct-Form implementation for a lter length M = 5 is shown in Fig. 2.
z
-1
X
b (n)
z
-1
X
b (n)
+
+
z
-1
X
b (n)
+
+
z
-1
X
b (n)
+
X
b (n)
+
+
+ +
0 1 2 3 4
y(n)
d(n)
e(n)
Adapti ve LMS Al gori thm
b (n+1) = b (n) + e(n)f(n-k)
Figure 2: A Direct-Form LMS adaptive lter with length M = 5..
2.2 Expanded Derivation
For a lter as shown in Fig. 1, the mean-squared-error (MSE) is dened as
MSE = E e
2
(n) = E (d(n) y(n))
2
= E d
2
(n) + E y
2
(n) 2E {d(n)y(n)}
=
dd
(0) +
yy
(0) 2
dy
(0) (5)
where E {} is the expected value,
dd
(k) and
yy
(k) are autocorrelation functions and
dy
(k) is the
cross-correlation function between {d(n)} and {y(n)}.
The lter output is
M 1
y(n) = b
k
f(n k)
k=0
and for stationary waveforms, Eq. (5) at time step n reduces to
M1 M1 N1
MSE =
dd
(0) + b
m
(n)b
k
(n)
ff
(m k) 2 b
k
(n)
fd
(k) (6)
m=0 k=0 k=0
2
2.2.1 The Optimal FIR Coecients
The optimal FIR lter coecients b
opt
, k = 0, . . . , M 1, that minimize the MSE, are found by
k
setting the derivatives with respect to each of the b
k
(n)s equal to zero. From Eq. (6)
(MSE)
= 2
N1

=0 m
b
opt

ff
(m k) 2
fd
(k) = 0, k = 0, 1, . . . , N 1 (7)
m
b
k
(n)
which is a set of linear equations in the optimal coecients b
opt
k
, and which in matrix form is written
Rb
opt
= P or b
opt
= R
1
P, (8)
where
_

f f
(0)

f f
(1)

f f
(2)
.
.
.

ff
(1)

ff
(0)

ff
(1)
.
.
.

ff
(2)

ff
(1)

ff
(0)
.
.
.

ff
(M 1)

ff
(M 2)

ff
(M 3)
.
.
.

f f
(M 1)
ff
(M 2)
ff
(M 3)
ff
(0)
_

R =
_ _
is a Toeplitz matrix, known as the correlation matrix, and
P = [
fd
(0)
fd
(1)
fd
(2)
fd
(M 1)]
T

is the cross-correlation vector.
With these denitions the MSE, as expressed in Eq. (6), reduces to
MSE =
dd
(0) + b(n)
T
Rb(n) 2P
T
b(n), (9)
and the minimum MSE is
MSE
min
=
dd
(0) P
T
b
opt
(10)
2.2.2 The LMS Algorithm
Assume an update algorithm of the form
b(n + 1) = b(n) +
1
2
(n)S(n) (11)
where S(n) is a direction vector that will move b(n) toward b
opt
, and (n) is an empirically chosen
gain schedule that determines the step-size at each iteration. In particular, with the method of
steepest descent, let S(n) = g(n) where
d(MSE)
g(n) =
db
is the gradient vector, and
MSE
g
k
(n) = = 2
1

M
b
k
m=0
b
k
(n)
ff
(m k) 2
fd
(k), k = 0, 1, . . . M 1.
from Eq. (6). Then, as above,
g(n) = 2 [Rb(n) P] , (12)
and the LMS algorithm is
b(n + 1) = b(n)
1
2
(n)g(n) (13)
= [I (n)R] b(n) + (n)P (14)
3
where I is the identity matrix. It is interesting to note that if we dene
b(n) = b
opt
b(n)
the LMS algorithm becomes
b(n + 1) = b(n) (n)Rb(n) (15)
and if any b
k
(n) = b
opt
k
, b
k
(n + 1) = b
opt
k
for all subsequent time steps.
In practice the LMS algorithm does not have P or R available, and we seek an estimator for
[Rb(n) P]. The error e(n) is
M1
e(n) = d(n) y(n) = d(n)

b(k)f(n k)
k=0
and therefore

M1

b(k)f(n j)f(n k)

, for j = 0, 1, 2, . . . , M1. E {e(n)f(n j)} = E {d(n)f(n j)}E
k=0
The individual equations may be collected into vector form
E {e(n)f (n)} = P Rb(n) (16)
and using Eq. (12) the gradient vector can be written
g(n) = 2E {e(n)f (n)} . (17)
An unbiased estimate g(n) of the gradient vector at the nth iteration is simply found from Eq, (17)
as
g(n) = 2e(n)f (n), (18)
and substituted into Eq, (11) to generate the LMS algorithm
b(n + 1) = b(n) + (n)e(n)f (n), (19)
which is identical to that obtained with the simplied derivation in Eq. (4).
2.3 Convergence Properties
A full discussion of the convergence properties of the LMS algorithm is beyond the scope of this
handout The value of the gain constant must be selected with some care. We simply state without
proof that b(n) will converge to b
opt
provided
2
0 < < , for k = 0, 1, 2, . . . , M 1

k
where
k
is an eigenvalue of the matrix R. Because R is an auto-correlation matrix, its eigenvalues
are non-negative and
M1

max
<

k
= traceR = M
ff
(0) = E

f
2
(n)

k=0
To ensure stability
2
< (20)
M.E {f
2
(n)}
If is too large it will cause the system to overshoot the optimal values for the b(n) and the
system will become unstable. The convergence rate is dependent on the ratio of the minimum to
maximum eigenvalues of R. If
min
/
max
1, the convergence will be fast, but conversely, if

min
/
max
<< 1 the convergence will be sluggish, and the lter will not track rapidly changing
conditions.
4
2.4 Variations on the Basic LMS Algorithm
2.4.1 Fixed Gain Schedule Implementation
The LMS algorithm is commonly used with a xed gain schedule (n) = for two reasons: rst in
order that the lter can respond to varying signal statistics at any time it is important that, (n)
not be a direct function of n. If (n) 0 as n , adaptation could not occur. The second
factor is that the xed gain LMS algorithm is easy to implement in hardware and software.
2.4.2 The Normalized MLS Algorithm
Equation (20) demonstrates that the convergence and stability depend of the signal power. To
normalize this dependence a modied form of the LMS algorithm, frequently used in practice is

b(n + 1) = b(n) +
2
e(n)f (n), (21)
||f(n)||
which is essentially a variable gain method with

(n) =
2
||f(n)||
To avoid numerical problems when the norm of the signal vector is small, a small positive
constant is often added

(n) =
2
+ ||f(n)||
These algorithms are known as normalized MLS, or NMLS, algorithms.
2.4.3 Smoothed MLS Algorithms
Because the update algorithm uses an estimator of the gradient vector the lter coecients will be
subject to random perturbations even when nominally converged to b
opt
. Several approaches have
been used to smooth out these uctuations:
(a)Use a running average over the last K estimates.
A FIR lter may be used to provide a smoothed estimate g(n) of the gradient vector, g(n),
for example a xed length moving average
1
K1
g(n) =

g(n k). (22)
K
k=0
The LMS algorithm (Eq. (13)) is then based on the averaged gradient vector eatimate:
1
b(n + 1) = b(n) (n)g(n)
2
(b)Update the coecients every K steps, using the average.
A variation on the above is to update the coecients every N time steps, and compute the
average of the gradient vector estimates during the intermediate time steps.
1
N1
g(Nn) =

g(nN + k). (23)
K
k=0
The update equation, applied every N time-steps, is
1
b((n + 1)N) = b(nN) (Nn)g(Nn)
2
5
(c)Use a simple IIR lter to smooth the estimate of the gradient vector.
The noisy estimates of the gradient vector, Eq. (18), may be smoothed with a simple rst-
order, unity-gain, low-pass IIR lter with a transfer function
H
s
(z) =
1
1 z
1
to produce a smoothed estimate g(n) of g(n). The dierence equation is:
g(n) = g(n) + (1 )g(n). (24)
The value of controls the degree of smoothing. If = 0 then g(n) = g(n) and there is no
smoothing, but as 1 the contribution from the most recent estimate decreases and the
smoothing increases. As above, the LMS algorithm (Eq. (13)) is then
1
b(n + 1) = b(n)
2
(n)g(n)
2.4.4 Implementations Based on the Sign of the Error
For hardware implementations, where multiplication operations are computationally expensive, it
may be possible to use steps that are related to the sign of all or part of the gradient vector estimate,
for example three possibilities are:
b(n + 1) = b(n) + sgn(e(n))f (n) (25)
b(n + 1) = b(n) + e(n)sgn(f (n)) (26)
b(n + 1) = b(n) + sgn(e(n))sgn(f (n)) (27)
where sgn() is the signum function. In the last case numerical multiplication can be eliminated
completely. Care must be taken to ensure stability and convergence with such reduced complexity
implementations.
3 A MATLAB Demonstration Adaptive Least-Squares Filter
% -------------------------------------------------------------------------
% 2.161 Classroom Example - LSadapt - Adaptive Lleast-squares FIR filter
% demonstration
% Usage : 1) Initialization:
% y = LSadapt(initial, Lambda, FIR_N)
% where Lambda is the convergence rate parameter.
% FIR_N is the filter length.
% Example:
% [y, e] = adaptfir(initial, .01, 51);
% Note: LSadapt returns y = 0 for initialization
% 2) Filtering:
% [y, b] = adaptfir(f, d};
% where f is a single input value,
% d is the desired input value, and
% y is the computed output value,
% b is the coefficient vector after updating.
%
% Version: 1.0
6
% Author: D. Rowell 12/9/07
% -------------------------------------------------------------------------
%
function [y, bout] = LSadapt(f, d ,FIR_M)
persistent f_history b lambda M
%
% The following is initialization, and is executed once
%
if (ischar(f) && strcmp(f,initial))
lambda = d;
M = FIR_M;
f_history = zeros(1,M);
b = zeros(1,M);
b(1) = 1;
y = 0;
else
% Update the input history vector:
for J=M:-1:2
f_history(J) = f_history(J-1);
end;
f_history(1) = f;
% Perform the convolution
y = 0;
for J = 1:M
y = y + b(J)*f_history(J);
end;
% Compute the error and update the filter coefficients for the next iteration
e = d - y;
for J = 1:M
b(J) = b(J) + lambda*e*f_history(J);
end;
bout=b;
end
4 Application - Suppression of Narrow-band Interference in a
Wide-band Signal
In this section we consider an adaptive lter application of suppressing narrow band interference,
or in terms of correlation functions we assume that the desired signal has a narrow auto-correlation
function compared to the interfering signal.
Assume that the input {f(n)} consists of a wide-band signal {s(n)} that is contaminated by a
narrow-band interference signal {r(n)} so that
f(n) = s(n) + r(n).
The ltering task is to suppress r(n) without detailed knowledge of its structure. Consider the
lter shown in Fig. 3. This is similar to Fig. 1, with the addition of a delay block of time steps
in front of the lter, and the denition that d(n) = f(n). The overall ltering operation is a little
unusual in that the error sequence {e(n)} is taken as the output. The FIR lter is used to predict
the narrow-band component so that y(n) r(n), which is then subtracted from d(n) = f(n) to
leave e(n) s(n).
7

causal l i near F R fi l ter
H(z)
Z
,
f
f
y
n
n-, n
+
-
error
del ay
Adapti ve Least-Squares
Al gori thm
d
n
fi l ter coeffi ci ents
s
r
n
n
narrow-band
i nterference
wi de-band
si gnal
e s
n n
Figure 3: The adaptive least-squares lter structure for narrow-band noise suppression.
The delay block is known as the decorrelation delay. Its purpose is to remove any cross-
correlation between {d(n)} and the wide-band component of the input to the lter {s(n )}, so
that it will not be predicted. In other words it assumes that

ss
() = 0, for > . | |
This least squares structure is similar to a -step linear predictor. It acts to predict the current
narrow-band (broad auto-correlation) component from the past values, while rejecting uncorrelated
components in {d(n)} and {f(n )}.
If the LMS lter transfer function at time-step n is H
n
(z), the overall suppression lter is FIR
with transfer function H(z):
H(z) =
E(z)
=
F (z) z

H
n
(z)F (z)
F (z) F (z)
= 1 z

H
n
(z)
= z
0
+ 0z
1
+ . . . + 0z
(1)
b
0
(n)z

b
1
(n)z
(+1)
+ . . .
. . . b
M1
(n)z
(+M1)
(28)
that is, a FIR lter of length M + with impulse response h

(k) where
h

_

_
_

(k) =
1 k = 0
0 1 k < (29)
k M + 1 b
k
(n)
and with frequency response
M+1
k=0
The lter adaptation algorithm is the same as described above, with the addition of the delay ,
that is
b(n + 1) = b(n) + e(n)f (n )) (31)
or
b
k
(n + 1) = b
k
(n) + e(n)f((n ) k), k = 0, 1, 2, . . . M 1. (32)
4.1 Example 1: Demonstration of Convergence with a Sinusoidal Input
In the handout MATLAB Examples of Least-Squares FIR Filter Design, example A One-step Linear
Predictor for a Sinusoidal Input we examined the static least-squares lter design for the case
H(e
j
) = h

(k)e
jk
(30) .
8
described by Stearns and Hush for the one-step predictor of a sine wave with lengths M = 2 and
M = 3. In this rst example, we note the similarity of the adaptive lter to the one-step predictor
and examine the convergence of the lter to the closed-form lter coecients. The input is a
noise-free sinusoid

2n

f(n) = sin .
12
The stability of the algorithm is governed by Eq. (20), and since for a sinusoid E

f(n)
2

= 0.5 we
are constrained to
2

2 for M = 2
< <
M.E {f(n)
2
} 4/3 for M = 3
The following MATLAB script was used:
% Example - Demonstration of convergence with a sinusoidal input
for J=1:1000
f(J) = sin(2*pi*J/12);
end
% Initialize the filter with M = 2, Delta =1
% Choose filter gain parameter Lambda = 0.1
Delta = 1;
Lambda = 0.1;
M = 2;
x = LSadapt(initial,Lambda,M);
% Filter the data
y = zeros(1,length(f));
e = zeros(1,length(f));
f_delay = zeros(1,Delta+1);
% Filter - implement the delay
for J = 1:length(f)
for K = Delta+1:-1:2
f_delay(K) = f_delay(K-1);
end
f_delay(1) = f(J);
% The desired output is f(J), the filter input is the delayed signal.
[y(J),b_filter] = LSadapt(f_delay(Delta+1),f(J));
end;
The script was modied and run with M = 2 and 3, and with various values of . The convergence
of the error is demonstrated in Fig. 4. The dependence of the convergence upon M and is clearly
demonstrated.
The values reported for the lter coecients with M = 2 were
b(0) = 1.73117, b(1) = 1
which are in agreement with the solution b(0) =

3, and b
1
= 1. For M = 3 the values returned
were
b(0) = 1.24290, b(1) = 0.15275, b(2) = 0.48916.
As Stearns and Hush note, there is no unique solution for the coecients for M = 3, but the
optimal lter must satisfy the conditions:
b(0) b(1) =

3, b(0) +

3b(1) + 2b(2) = 0
The reported values satisfy these constraints, indicating that the lter has found an optimal solu-
tion.
9
0 100 200 300 400 500 600 700 800 900 1000
0.5
0
0.5
Predictor Error (M=2 Lambda=0.1)
Time step (n)
E
r
r
o
r
0 100 200 300 400 500 600 700 800 900 1000
0.5
0
0.5
Predictor Error (M=3 Lambda=0.1)
Time step (n)
E
r
r
o
r
0 100 200 300 400 500 600 700 800 900 1000
0.5
0
0.5
Predictor Error (M=2 Lambda=1)
E
r
r
o
r
Time step (n)
Figure 4: Convergence of predictor error e(n) with lter length M and gain .
4.2 Example 2: Suppression of a Sinusoid in Noise
For the second example we look at the rejection of a sinusoid of unknown frequency in white noise.
This case is extreme in that the signal {s(n)} has an auto-correlation function
ss
(n) = (n), while
the interference has a periodic auto-correlation.
The following MATLAB script demonstrates the ecacy of the method.
% Create the input as white noise with a strong sinusoidal component
f = randn(1,10000);
L = length(f);
y = zeros(1,length(f));
e = zeros(1,length(f));
Lambda = .001; Delta = 1; M=15;
x = LSadapt(initial, Lambda, M);
f_delay = zeros(1,Delta+1);
for J = 1:L
f(J) = f(J) + 3*sin(2*pi*J/12);
for K = Delta+1:-1:2
f_delay(K) = f_delay(K-1);
end
f_delay(1) = f(J);
[y(J),b] = LSadapt(f_delay(Delta+1),f(J));
e(J) = f(J) - y(J);
end;
w=-pi:2*pi/1000:pi;
10
subplot(1,2,1), plot(w, fftshift(abs(fft(f(L-1000:L)))));
xlabel(Normalized frequency)
ylabel(Magnitude)
title(Input spectrum)
subplot(1,2,2), plot(w, fftshift(abs(fft(e(L-1000:L)))));
xlabel(Normalized frequency)
ylabel(Magnitude)
title(Output spectrum)
Figure 5 shows the input and output spectra of the last 1000 samples of the data record.
Inputspectrum Outputspectrum
0
200
400
600
800
1000
1200
M
a
g
n
i
t
u
d
e

0
200
400
600
800
1000
1200
M
a
g
n
i
t
u
d
e

0 1 2 3 0 1 2 3
Normalizedfrequency Normalizedfrequency
Figure 5: Input and output spectra for the lter in Example 2.
4.3 Example 3: Frequency Domain Characteristics of an LMS Suppression Fil-
ter
This example demonstrates the lter characteristics after convergence. The interfering signal is
comprised of 100 sinusoids with random phase and random frequencies between 0.3 and 0.6. The
signal is white noise. The lter used has M = 31, = 1, and was adjusted to give a reasonable
convergence rate. The overall system H(z) = 1 z

H
n
(z) frequency response magnitude, Eq.
(30), is then computed and plotted, along with the z-plane pole-zero plot.
% The frequency domain filter characteristics of an interference
% suppression filter with finite bandwidth interference
%
% Create the interference as a closely packed sum of sinusoids
% between 0.3pi < Omega < 0.6pi with random frequency and phase
11
phase = 2*pi*rand(1,100);
freq = 0.3 + 0.3*rand(1,100);
f = zeros(1,100000);
for J=1:100000
f(J) = 0;
for k = 1:100
f(J) = f(J) + sin(freq(k)*J + phase(k));
end
end
% The "signal" is white noise
signal = randn(1,100000);
f = .005*f + 0.01*signal;
% Initialize the filter with M = 31 , Delta =1
% Choose filter gain parameter Lambda = 0.1
Delta = 1; Lambda = 0.5; M = 31;
x = LSadapt(initial,Lambda, M);
% Filter the data
f_delay = zeros(1,Delta+1);
y = zeros(1,length(f));
e = zeros(1,length(f));
for J = 1:length(f)
for K = Delta+1:-1:2
f_delay(K) = f_delay(K-1);
end
f_delay(1) = f(J);
[y(J),b] = LSadapt(f_delay(Delta+1),f(J));
e(J) = f(J) - y(J);
end;
% Compute the overall filter coefficients
% H(z) = 1 - z^{-Delta}H_{LMS}(z)
b_overall = [1 zeros(1,Delta-1) -b];
% Find the frequency response
[H,w] = freqz(b_overall,1);
zplane(b_overall,1)
The input and output spectra are shown in Fig. 6, and the lter frequency response magnitude is
shown in Fig. 7. The adaptive algorithm has clearly generated a notch-lter covering the bandwidth
of the interference. The pole-zero plot in Fig. 8 shows how the zeros have been placed over the
spectral region (0.3 < < 0.6) to create the band-reject characteristic..
4.4 Example 4: Suppression of a Sliding Sinusoid Superimposed on a Voice
Signal
In this example we demonstrate the suppression of a sinusoid with a linearly increasing frequency
superimposed on a voice signal. The ltering task is to task is to suppress the sinusoid so as to
enhance the intelligibility of the speech. The male voice signal used in this example was sampled
at F
s
= 22.05 kHz for a duration of approximately 8.5 sec. The interference was a sinusoid

2

F
s
2

r(t) = sin((t)) = sin t + t
150
12
0 1 2 3
0
1
2
3
4
5
6
7
8
Normalized angular frequency
M
a
g
n
i
t
u
d
e
Spectrum of input signal f(n)
0 1 2 3
0
1
2
3
4
5
6
7
8
Normalized angular frequency
M
a
g
n
i
t
u
d
e
Spectrum of output signal e(n)
Figure 6: Input and output spectra from an adaptive suppression lter with interference in the
band 0.3 < < 0.6.
0 0.5 1 1.5 2 2.5 3
30
25
20
15
10
5
0
5
Adaptive Filter Frequency Response
Normalized frequency
M
a
g
n
i
t
u
d
e

(
d
B
)
Figure 7: Frequency response magnitude of an adaptive suppression lter with interference in the
band 0.3 < < 0.6.
13
AdaptiveFilterzplanepole/zeroplot
1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
1
31
I
m
a
g
i
n
a
r
y

P
a
r
t

1 0.5 0 0.5 1
RealPart
Figure 8: Pole/zero plot of an adaptive suppression lter with interference in the band 0.3 < <
0.6.
where F
s
= 22.05 kHz is the sampling frequency. The instantaneous angular frequency (t) =
d(t)/dt is therefore
(t) = 2(50 + 294t) rad/s
which corresponds to a linear frequency sweep from 50 Hz to approx 2550 Hz over the course of
the 8.5 second message. In this case the suppression lter must track the changing frequency of
the sinusoid.
% Example 2: Suppression of a frequeny modulated sinusoid superimposed on speech.
% Read the audio file and add the interfering sinusoid
[f,Fs,Nbits] = wavread(crash);
for J=1:length(f)
f(J) = f(J) + sin(2*pi*(50+J/150)*J/Fs);
end
wavplay(f,Fs);
% Initialize the filter
M = 55; Lambda = .01; Delay = 10;
x = LSadapt(initial, Lambda, M);
y = zeros(1,length(f));
e = zeros(1,length(f));
b = zeros(length(f),M);
f_delay = zeros(1,Delay+1);
% Filter the data
for J = 1:length(f)
for K = Delta+1:-1:2
f_delay(K) = f_delay(K-1);
end
f_delay(1) = f(J);
[y(J),b1] = LSadapt(f_delay(Delta+1),f(J));
e(J) = f(J) - y(J);
b(J,:) = b1;
14
0
500
1000
1500
2000
2500
M
a
g
n
i
t
u
d
e

end;
%
wavplay(e,Fs);
The script reads the sound le, adds the interference waveform and plays the le. It then lters
the le and plays the resulting output. After ltering the sliding sinusoid can only be heard very
faintly in the background. There is some degradation in the quality of the speech, but it is still
very intelligible.
Figs. 9 and 10 show the waveform spectra before and after ltering. Figure 9 clearly shows the
superposition of the speech spectrum on the pedestal spectrum of the swept sinusoid. The pedestal
has clearly been removed in Fig. 10. Figure 11 shows the magnitude of the frequency response lter
as a meshed surface plot, with time as one axis and frequency as the other. The rejection notch is
clearly visible, and can be seen to move from a low frequency at the beginning of the message to
approximately 2.5 kHz at the end.
InputSpectrum
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000
Frequency(Hz)
Figure 9: Example 4: Input spectrum of a sliding sinusoid superimposed on a speech waveform.
5 Application - Adaptive System Identication
An adaptive LMS lter may be used for real-time system identication, and will track slowly varying
system parameters. Consider the structure shown in Fig. 12. A linear system with an unknown
impulse response is excited by wide-band excitation f(n), The adaptive, length M FIR lter works
in parallel with the system, with the same input. Since it is an FIR lter, its impulse response is
the same as the lter coecients, that is
h(m) = b(m), for m = 0, 1, 2, . . . M 1.
and with the error e(n) dened as the dierence between the system and lter outputs, the minimum
MSE will occur when the lter mimics the system, at which time the estimated system impulse

h(m) response may be taken as the converged lter coecients.

FilteredOutputSpectrum
Frequency(Hz)
Figure 10: Example 4: Output spectrum from the adaptive least-squares ltering to enhance the
speech waveform.
5.1 Example
Consider a second-order unknown system with poles at z
1
, z
2
= Re
j
, that is with transfer
function
1
H(z) = ,
1 2Rcos()z
1
+ R
2
z
2
where the radial pole position R varies slowly with time. The following MATLAB script uses
LSadapt() to estimate the impulse response with 10,000 samples of gaussian white noise as the
input, while the poles migrate from z
1
, z
2
= 0.8e
j/5
to 0.95e
j/5
% Adaptive SysID
f = randn(1,10000);
% Initialize the filter with M = 2, Delta =.8
% Choose filter gain parameter Lambda = 0.1
Lambda = 0.01; M = 51;
x = LSadapt(initial,Lambda,M);
% Define the "unknown" system
R0 = .8; R1 = 0.95; ctheta = cos(pi/5);
delR = (R1-R0)/L;
L = length(f);
b=zeros(M,L);
ynminus2 = 0; ynminus1 = 0;
for J = 1:L
% Solve the difference equation to determine the system output at this iteration
R = R0 + delR*(J-1);
yn = 2*R*ctheta*ynminus1 - R^2*ynminus2 + f(J);
ynminus2 = ynminus1;
ynminus1 = y;
[yout,b(:,J)] = LSadapt(f(J),yn);
end;
Figure 13 shows the estimated impulse response,

h(m) = b(m), as the poles approach the unit circle

(
d
B
)

20
0
20
40
60
10
8
4000
6
3000
4
2000
2
1000
Time(sec) 0
0
Frequency(Hz)
Figure 11: Meshed surface showing the time dependence the frequency response magnitude. The
motion of the lter notch is seen as the valley in the response as the interference signals frequency
changes.
f(n)
unknown LT system system output
h(m)
d(n)
causal l i near F R fi l ter
y(n)
+
H(z)
-
esti mated i mpul se
response
fi l ter coeffi ci ents
h(m)
adapti ve Least-Squares
e(n)
al gori thm error
Figure 12: Adaptive ltering structure for system identication.
17
the changing system dynamics.
2
1.5
1
I
m
p
u
l
s
e

r
e
s
p
o
n
s
e

h
(
n
)
0.5
0
0.5
1
0.95
1.5
0.9
0
10
20
0.85
Poleradius
30
Timestep(n)
40
50
0.8
Figure 13: Estimated impulse response of a second-order system with dynamically changing poles
using an adaptive LMS lter (length 51) with white noise as the input.
18