Advance Mathematical Methods
Course Overview
Machine Learning and Data Science are being used these days in a variety of applications including, but
not limited to, forecasting in economics and finance, predicting anomalies or signal analysis in
engineering, identification of speaker in acoustics, detection of cosmic bubbles in astrophysics and
diagnosis in medical imaging.
While machine learning and data science have enabled many success stories, and tools are readily
available to analyze data or design machine learning systems, the strong mathematical foundations in
these areas are of significant importance to understand, review, analyze and evaluate the technical
details of the machine learning systems and data science algorithms that are usually abstracted away
from the user. This course focuses on the mathematical foundations that are essential to build an
intuitive understanding of the concepts related to Machine Learning and Data Science.
Topics covered are
Linear Algebra: vectors and matrices, vector spaces, system of linear equations, eigen-value
decomposition, singular value decomposition, regression, least-squares, regularization
Calculus: Multivariate calculus and differentials for optimization, gradient descent
Probability: probability axioms, Bayes rule, random variable, probability distributions
Statistics: descriptive stats, inferential stats, statistical tests
Introduction to Neural Networks: single and multi-layer perceptron(s), feedforward and
feedback networks
Application to machine learning and data science: principal component analysis (PCA), time
series forecasting, clustering etc
Hands-on exercises: Implementation of the exercises will be carried out in Python
COURSE OBJECTIVES
The goal of this course is to provide mathematical foundations of Machine Learning and Data Science. In
broad brush, this course aims to:
Provide a thorough introduction to both fundamental and advanced topics of linear
algebra necessary for machine learning and data science
Build mathematical foundations of calculus, probability and statistics
Provide an appreciation for applications of ML and Data Science
Equip the students with the basics of Python to enable them to implement and evaluate
Machine Learning and Data Science algorithms
Learning Outcomes
The students should be able to:
CLO1: Understand the core theoretical concepts serve as foundations of Machine Learning and Data
Science
CLO2: Understand the core theoretical concepts of calculus, probability theory and statistics that serve
as foundations of Machine Learning and Data Science
CLO3: Formulate and implement basic problems in machine learning and data science
Textbook(s)/Supplementary Readings
S.Boyd and L. Vandenberghe. Introduction to Applied Linear Algebra – Vectors, Matrices, and
Least Squares. Cambridge University Press, 2019
M. P. Deisenroth, A. A. Faisal and Cheng Soon Ong. Mathematics for Machine Learning.
Cambridge University Press, 2019
(https://www.google.com.pk/books/edition/Mathematics_for_Machine_Learning/
pFjPDwAAQBAJ?hl=en&gbpv=1&printsec=frontcover )
G. Strang. Introduction to Linear Algebra. 2016
J. A. Gubner, Probability and Random Processes for Electrical and Computer Engineers,
Cambridge University Press, 2006.
S. L. Miller and D. Childers, Probability and Random Processes: With Applications to Signal
Processing and Communications.
Papoulis and S.U. Pillai, Probability, Random Variables, and Stochastic Processes.
Course Topics
Module Topic Additional Remarks
Course Overview, notation, vectors and
matrices, basic operation on vectors
Tutorial 1: Basic matrix and
Advanced operations on vectors, norm, angle, vector operations
inner product
Lab exercise 0: Intro to Python
Basic Linear
Operations on matrices Lab exercise 1: Linear
Algebra
independence, basis, matrix rank
Linear independence, basis, matrix rank
Matrix vector product interpretation
Systems of linear equations, matrix inverses, Tutorial 2: Advanced matrix and
pseudo inverse vector operations
Least-squares, constrained least squares Tutorial 3: Solving a system of
regularization linear equations, least-squares
Advanced Linear regularization
Least-squares application: data-fitting, clustering
Algebra
Lab exercise 2: Least-squares
application: data-fitting
Eigenvalue decomposition plus geometric
interpretation Lab exercise 3: Eigen value
decomposition
Singular-value decomposition (SVD) plus
geometric interpretation
Intro to Calculus, interpretation of derivative and Tutorial 4: Interpretation:
first-order difference of the data derivative, integration, weighted
Calculus
average, moving average
Interpretation of integration, weighted average,
moving average of time-series
Introduction to probability theory, axioms of Tutorial 5: Axioms of probability
probability
Lab exercise 4: Probability
Bayesian analysis overview distributions and Statistical
Probability &
Inference, Statistical tests
Statistics
interpretation of output
Random variables and probability distributions
Introduction to statistical inference
Statistical tests
Introduction to Introduction to neural network. Single layer Tutorial 6: Hands-on working:
Neural Networks perceptron single layer perceptron example
Multi-layer perceptron, feedforward and
feedback networks, back propagation
Dimensionality Reduction using Principal
Component Analysis
Linear Regression, Time-series forecasting
Applications Lab exercise 5: Applications: PCA
and Classification
Classification: Perceptron classifier, Logistic
Regression
Clustering: k-means clustering
