Institute of Space Technology
Lesson Plan
Doc #
Date of Issue
CS No:
Page No.
IST-F-32/03
06-Aug-2013
DPTTSUBJECT-LP
RevNo
1 of 9
DEPARTMENT & PROGRAM: Electrical Engineering, Masters in Wireless Communications
COURSE CODE-COURSE NAME: EE-623 Adaptive Filters.
CR: 3
COURSE DESCRIPTION:
Students attending this lecture should learn the basics of adaptive filters. To achieve this, necessary
algorithms will be derived and applied to problems arising in speech and audio processing. The
algorithms comprise Wiener filtering, linear prediction, and adaptive schemes such as the NLMS
algorithm, affine projection, and the RLS algorithm. For applications from speech and audio
processing we use noise and reverberation reduction, echo cancellation, and beamforming.
PREREQUISITE:
EE-771 Stochastic Processes
CO-REQUISITE:
None
TEXT AND MATERIALS:
Textbook
ADAPTIVE FILTERS THEORY AND APPLICATIONS
Second Edition, Behrouz Farhang-Boroujeny, University of Utah, USA
References Material:
1. Adaptive Filters by Ali H.Sayed, 2008
2. Adaptive Filter Theory by Simon Haykin, 2002
3. Adaptive Filtering Algorithms and Practical Implementation Fourth Edition by Paulo S. R. Diniz
RELEVANT PROGRAM LEARNING OUTCOME:
The course is designed so that students will achieve the PLO/s:
1. Design/Development of Solution: Design solution for complex Networks problems and design
Networks/Systems, components or processes that meets specified needs with appropriate consideration for
public health and safety, cultural societal and environmental considerations.
2. Investigation: Conduct investigation into complex problems using research based methods including
design experiments, analysis and interpretation of data, and synthesis of information to provide valid
conclusions.
Course Objectives:
The aim of the course is to introduce techniques in adaptive signal processing and adaptive filter theory with
applications on related fields. Also the objective of this course is to provide the mathematical framework for
an understanding of adaptive statistical signal processing, where the goal is to extract information from noisy
or corrupted data, where the properties of the signal and/or the interference are partially unknown or change
Institute of Space Technology
Lesson Plan
Doc #
Date of Issue
CS No:
Page No.
IST-F-32/03
06-Aug-2013
DPTTSUBJECT-LP
RevNo
2 of 9
with time. The basic tools of vector spaces and discrete-time stochastic processes are reviewed and applied to
the methods of Wiener filtering and least-squares filtering. Various types of adaptive filters will be introduced
and their properties will be studied, specifically convergence, tracking, robustness and computational
complexity. Applications will mainly be addressed through student MATLAB based projects.
Course Learning Outcomes:
This course treats adaptive signal processing algorithms for extracting relevant information from noisy
signals. The emphasis is on recursive, model based estimation methods for signals and systems whose
properties change in time. Applications in, for example, communications, control and medicine are discussed.
Upon successful completion of the course, the student will be able to:
1. Identify applications in which it would be possible to use the different adaptive filtering approaches.
Analyze the accuracy and determine advantages and disadvantages of each method.
2. Design and apply optimal minimum mean square estimators and in particular linear estimators. To
understand and compute their expected performance and verify it. Design, implement and apply
Wiener filters (FIR, non-causal, causal) and evaluate their performance. Design, implement and apply
LMS and RLS filters for given applications.
3. Use a combination of theory and software implementations to solve adaptive signal problems. Use the
theoretical understanding to do troubleshooting, e.g., in cases the observed performance is not as
expected.
4. Report the solution and results from the application of the adaptive filtering techniques to given
problems. Implement and apply LMS and RLS filters for given applications.
PRACTICAL APPLICATIONS
The practical applications of this course are as follows
1. System identification
2. Linear predictor
3. Inverse modeling
4. Jammer suppression
5. Adaptive notch filter
6. Noise canceller
7. Echo cancellation
8. Voice echo canceller
9. Data echo canceller
10. Multiple-input multiple-output (MIMO) echo cancellation
11. Adaptive feedback cancellation in hearing aids
12. Fetal monitoring, cancelling of maternal ECG during labor
Institute of Space Technology
Lesson Plan
Doc #
Date of Issue
CS No:
Page No.
LECTURE PLAN:
Instruction
Discussion
Project
Exercises/Tutorial
3 of 9
70%
10%
10%
10%
Module
I.
IST-F-32/03
06-Aug-2013
DPTTSUBJECT-LP
RevNo
Topic
1. Introduction
Reference
Week/Lecture
Chapter 1
1/1-2
1.1
1.2
1.3
1.4
Linear Filters
Adaptive Filters
Adaptive Filter Structures
Adaptation Approaches
1.4.1 Approach Based on
Wiener Filter Theory
1.4.2 Method of Least-Squares
1.5 Real and Complex Forms of
Adaptive Filters
II.
1.6 Applications
1.6.1 Modeling
1.6.2 Inverse Modeling
1.6.3 Linear Prediction
1.6.4 Interference Cancellation
Chapter
1
2/3-4
III.
3 Wiener Filters
Chapter 3
3/5-6
3.1 Mean-Squared Error Criterion
3.2 Wiener Filter Transversal, RealValued Case
3.3 Principle of Orthogonality
3.4 Normalized Performance
Function
3.5 Extension to Complex-Valued
Case
3.6 Unconstrained Wiener Filters
3.6.1 Performance Function
3.6.2 Optimum Transfer
Function
3.6.3 Modeling
3.6.4 Inverse Modeling
3.6.5 Noise Cancellation
Institute of Space Technology
Lesson Plan
Doc #
Date of Issue
CS No:
Page No.
IV.
4 Eigenanalysis and
Performance Surface
IST-F-32/03
06-Aug-2013
DPTTSUBJECT-LP
RevNo
4 of 9
Chapter 4
4/7-8
Chapter 5
5/9-10
Chapter 6
6/11-12
Chapter 6
7/13-14
4.1 Eigenvalues and Eigenvectors
4.2 Properties of Eigenvalues and
Eigenvectors
4.3 Performance Surface
V.
5 Search Methods
5.1 Method of Steepest Descent
5.2 Learning Curve
5.3 Effect of Eigenvalue Spread
5.4 Newtons Method
5.5 An Alternative Interpretation of
Newtons Algorithm
VI.
VII.
6 LMS Algorithm
6.1 Derivation of LMS Algorithm
6.2 Average Tap-Weight Behavior of
the LMS Algorithm
6.3 MSE Behavior of the LMS
Algorithm
6.3.1 Learning Curve
6.3.2 Weight-Error Correlation
Matrix
6.3.3 Excess MSE and
Misadjustment
6.3.4 Stability
6.3.5 The Effect of Initial
Values of Tap Weights on the
Transient Behavior
6 LMS Algorithm
6.5 Simplified LMS Algorithms
6.6 Normalized LMS Algorithm
6.7 Affine Projection LMS Algorithm
6.8 Variable Step-Size LMS Algorithm
6.9 LMS Algorithm for ComplexValued Signals
6.10 Beamforming (Revisited)
6.11 Linearly Constrained LMS
Algorithm
6.11.1 Statement of the
Institute of Space Technology
Lesson Plan
Doc #
Date of Issue
CS No:
Page No.
VIII.
Problem and Its Optimal
Solution
6.11.2 Update Equations
IST-F-32/03
06-Aug-2013
DPTTSUBJECT-LP
RevNo
5 of 9
Chapter 9
8/15-16
Chapter
10
9/17-18
9 Subband Adaptive Filters
IX.
9.1 DFT Filter Banks
9.1.1 Weighted OverlapAdd Method
for Realization of DFT Analysis Filter
Banks
9.1.2 Weighted OverlapAdd Method
for Realization of DFT Synthesis Filter
Banks
9.2 Complementary Filter Banks
9.3 Subband Adaptive Filter Structures
9.4 Selection of Analysis and Synthesis
Filters
9.5 Computational Complexity
9.6 Decimation Factor and Aliasing
10 IIR Adaptive Filters
X.
10.1 Output Error Method
10.2 Equation Error Method
10.3 Case Study : IIR Adaptive Line
Enhancement
10.3.1 IIR ALE Filter, W(z)
10.3.2 Performance Functions
10.3.3 Simultaneous Adaptation of s
and w
10.3.4 Robust Adaptation of w
12.5.4 Excess MSE and
Misadjustment
12.5.5 Initial Transient Behavior of
the RLS Algorithm
14 Tracking
14.1 Formulation of the Tracking Problem
14.2 Generalized Formulation of LMS
Algorithm
14.3 MSE Analysis of the Generalized LMS
Algorithm
14.4 Optimum Step-Size Parameters
14.5 Comparisons of Conventional
10/19-20
Chapter
14
Institute of Space Technology
Lesson Plan
Doc #
Date of Issue
CS No:
Page No.
XI.
XII.
Algorithms
14.6 Comparisons Based on Optimum
Step-Size Parameters
14.7 VSLMS: An Algorithm with Optimum
Tracking Behavior
14.7.1 Derivation of VSLMS Algorithm
14.7.2 Variations and Extensions
14.7.3 Normalization of the Parameter
14.7.4 Computer Simulations
14.8 RLS Algorithm with Variable Forgetting
Factor
16 Active Noise Control
16.1 Broadband Feedforward SingleChannel ANC
16.1.1 System Block Diagram in the
Absence of the Secondary Path S1(z)
16.1.2 Filtered-X LMS Algorithm
16.1.3 Convergence Analysis
16.1.4 Adding the Secondary Path
S1(z)
IST-F-32/03
06-Aug-2013
DPTTSUBJECT-LP
RevNo
6 of 9
11/21-22
Chapter
16
12/23-24
Chapter
17
17 Synchronization and
Equalization in Data Transmission
Systems
XIII.
17.1 Continuous Time Channel Model
17.2 Discrete Time Channel Model and
Equalizer Structures
17.2.1 Symbol-Spaced Equalizer
17.2.2 Fractionally Spaced Equalizer
17.2.3 Decision Feedback Equalizer
17.3 Timing Recovery
17.3.1 Cost Function
17.3.2 The Optimum Timing Phase
17.3.3 Improving the Cost Function
17.3.4 Algorithms
17.3.5 Early-Late Gate Timing
Recovery
17.3.6 Gradient-Based Algorithm
17.4 Equalizers Design and Performance
Analysis
13/25-26
Chapter
18
Institute of Space Technology
Lesson Plan
Doc #
Date of Issue
CS No:
Page No.
XIV.
17.4.1 WienerHopf Equation for
Symbol-Spaced Equalizers
17.4.2 Numerical Examples
17.5 Adaptation Algorithms
IST-F-32/03
06-Aug-2013
DPTTSUBJECT-LP
RevNo
7 of 9
14/27-28
18 Sensor Array Processing
18.1 Narrowband Sensor Arrays
18.1.1 Array Topology and
Parameters
18.1.2 Signal subspace, noise
subspace, and spectral factorization
18.1.3 Direction of Arrival Estimation
18.1.4 Beamforming Methods
18.2 Broadband Sensor Arrays
18.2.1 Steering
18.2.2 Beamforming Methods
18.3 Robust Beamforming
18.3.1 Soft-Constraint Minimization
18.3.2 Diagonal Loading Method
18.3.3 Methods Based on Sample
Matrix Inversion
Chapter
19
19 Code Division Multiple
Access Systems
XV.
19.1 CDMA Signal Model
19.1.1 Chip-Spaced UsersSynchronous Model
19.1.2 Chip-Spaced UsersAsynchronous Model
19.1.3 Fractionally Spaced
Model
19.2 Linear Detectors
19.2.1 Conventional Detector:
The Matched Filter Detector
19.2.2 Decorrelator Detector
19.2.3 Minimum MeanSquared Error (Optimal)
Detector
19.2.4 Minimum Output
Energy (Blind) Detector
19.2.5 Soft Detectors
19.3 Adaptation Methods
19.3.1 Conventional Detector
15/29-30
Chapter
20
Institute of Space Technology
Doc #
Date of Issue
CS No:
Lesson Plan
Page No.
19.3.2
19.3.3
19.3.4
19.3.5
XVI.
Decorrelator Detector
MMSE Detector
MOE Detector
Soft Detectors
IST-F-32/03
06-Aug-2013
DPTTSUBJECT-LP
RevNo
8 of 9
16/31-32
20 OFDM and MIMO
Communications
20.1 OFDM Communication Systems
20.1.1 The Principle of OFDM
20.1.2 Packet Structure
20.1.3 Carrier Acquisition
20.1.4 Timing Acquisition
20.1.5 Channel Estimation
and Frequency Domain
Equalization
20.1.6 Estimation of Rhh and
R
20.1.7 Carrier-Tracking
Methods
20.1.8 Channel-Tracking
Methods
20.2 MIMO Communication Systems
20.2.1 MIMO Channel Model
20.2.2 Transmission
Techniques for SpaceDiversity Gain
20.2.3 Transmission
Techniques and MIMO
Detectors
for Space-Multiplexing Gain
20.2.4 Channel Estimation
Methods
20.3 MIMOOFDM
Review Week
Final Examination
16
COURSE TARGETS
Module
No.
CLO
No.
Teaching Methodology
Assessment Methodology
Learning Domain with
Level
Institute of Space Technology
Doc #
Date of Issue
CS No:
Lesson Plan
Page No.
I, II, IV,
XIII
III, V, VI,
X,
VII, VIII,
IX, XI
XII, XIV,
XV
IST-F-32/03
06-Aug-2013
DPTTSUBJECT-LP
RevNo
9 of 9
Lecture, Discussion
Quiz, Assignment, Exam
C1
Lecture, Discussion
Quiz, Assignment, Exam
C3
Quiz, Assignment, Exam
C2
Quiz, Assignment, Exam
C3
3
4
Lecture, Discussion,
Presentation
Lecture, Discussion,
Presentation
ASSESMENT:
Assignments
10%
Quizzes
15%
OHT Exam
30%
Project
15%
Final Exam
30%
_____________________________
Total
100%
Written By
Name with
Sign
Dr. Aamir Habib
Date
Reviewed By
Name with
Sign
Ch. Bilal A. Khan
Date
Approved By
Name with
Sign
Date
Dr. Qamar ul Islam
