Course Logistics and Introduction

CSN-382
Machine Learning
 2
Course Logistics
 Course Name: Machine Learning (CSN-382)
 An introductory course – supposed to be your first intro to the subject

 Three lectures every week

 All material will be posted on Microsoft Team for the course

 Course-related announcements will be sent on the class mailing list (and also
on Microsoft Teams)
 3
Grading

 Mid-term Exam: 25%

 End-term Exam: 50%

 Course Project: 25%

 4
Course Project: 25%

• Project to be done in groups of three students each

• Start forming your group from today onwards
• Will ask you to submit group details after one week
• Groups can only contain registered students (no auditors)
• You may propose your own project idea
• but please discuss the scope/feasibility with TAs
• Many types of projects possible (theoretical/applied/mixed); even building a
cool/useful app/portal for something using ML as its backend would be a nice
project. Explore around for ideas.
• Important: Don’t (re)use a project from another course you’ve done before (or
doing currently)
• Need to submit a formal project proposal
• Should contain a problem description, tentative plan of action, etc (I’ll provide more
guidelines later)
 5
Course Project: 25%

• Multiple Project presentations to assess the progress of the project

• Project evaluation will be done based on the novelty of the work
• Final Submission will include code + report
• Code should be in Python (preferably) – start learning Python today
• Report must be in LaTeX (preferably) – start learning LaTeX today
• Python Resources: several available –choose your favourite
• www.geeksforgeeks.org/python-programming-language/
• LaTeX resources: several available –choose your favourite
• www.overleaf.com/learn/latex/Tutorials
 6
Textbook and References
 Many excellent texts but none “required”. Some include:

 Different books might vary in terms of

 Set of topics covered
 Flavor (e.g., classical statistics, deep learning, probabilistic/Bayesian, theory)
 Terminology and notation (beware of this especially)
 We will provide you the reading material from the relevant sources
 Avoid using too many sources until you have developed a reasonable
understanding of a concept
 7
Textbook and References
• Hal Daumé III, A Course in Machine Learning (CIML), 2017 (freely available online)
• Kevin Murphy, Machine Learning: A Probabilistic Perspective (MLAPP), MIT Press, 2012
• Christopher Bishop, Pattern Recognition and Machine Learning (PRML), Springer, 2007.
• David G. Stork, Peter E. Hart, and Richard O. Duda. Pattern Classification (PC), Wiley-
Blackwell, 2000
• Ian Goodfellow and Yoshua Bengio and Aaron Courville. Deep Learning (DL), MIT Pess,
2016 (individual chapters freely available online)
• Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning
(ESL), Springer, 2009 (freely available online)
• Shai Shalev-Shwartz and Shai Ben-David.
Understanding Machine Learning: From Theory to Algorithms (UML), Cambridge University
Press, 2014
• Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar.
Foundations of Machine Learning (FOML), MIT Press, 2012
 8
Textbook and References
 Some of the course materials have been taken from the following links:

 https://www.cse.iitk.ac.in/users/piyush/courses/ml_autumn18/index.html

 https://github.com/purushottamkar/ml19-20w

 Thanks to Prof. Piyush for supporting in preparing the course material

 9
Course Goals..
 Introduction to the foundations of machine learning models and algos

 Focus on developing the ability to

 Understand the underlying principles behind ML models and algos
 Understand how to implement and evaluate them
 Understand/develop intuition on choosing the right ML model/algo for your problem

 (Hopefully) inspire you to work on and learn more about ML

 Not an introduction to popular software frameworks and libraries, such as

scikit-learn, PyTorch, Tensorflow, etc
 Can explore once you have some understanding of various ML techniques
Introduction to Machine
Learning
 11
Machine Learning (ML)
 Designing algorithms that ingest data and learn a model of the data
 The learned model can be used to
 Detect patterns/structures/themes/trends etc. in the data
 Make predictions about future data and make decisions

 Modern ML algorithms are heavily “data-driven”

 No need to pre-define and hard-code all the rules (infeasible/impossible anyway)
 The rules are not “static”; can adapt as the ML algo ingests more and more data
 12
ML: From What It Does to How It Does It?
 ML enables intelligent systems to be data-driven rather than rule-driven
 How: By supplying training data and building statistical models of data
 Pictorial illustration of an ML model for binary classification:

A Linear Classifier (the statistical model)

 13
ML: From What It Does to How It Does It?
 ML enables intelligent systems to be data-driven rather than rule-driven
 How: By supplying training data and building statistical models of data
 Pictorial illustration of an ML model for binary classification:

A Probabilistic Classifier (the statistical model)

P(“cat”|image) P(“dog”|image)
 14
Overfitting = Bad ML
 Doing perfectly on training data is not good enough

 A good ML model must generalize well on unseen (test data)

 Simpler models should be preferred over more complex ones!

 15
ML Applications Abound..

Picture courtesy: gizmodo.com,rcdronearena.com,www.wiseyak.com,www.charlesdong.com

 16
Key Enablers for Modern ML
 Availability of large amounts of data to train ML models

 Increased computing power (e.g., GPUs)

 17
ML: Some Success Stories
 18
ML: Some Success Stories
 19
ML: Some Success Stories

Picture courtesy: https://news.microsoft.com/

 20
ML: Some Success Stories
 Automatic Program Correction

Example from “Compilation error repair: for the student programs, from the student programs”, Ahmed et al (2018)
 21
Good ML Systems Should be Fair and Unbiased

 Good ML should not just be about getting high accuracies

 Should also ensure that the ML models are fair and unbiased
Criminals?

Not Criminals?

An image captioning system Don’t want a self-driving car Don’t want a predictive policing
should not always assume a that is more likely to hit black system that predicts criminality
specific gender in examples people than white people using facial features
like the above

 A lot of recent focus on Fairness and Transparency of ML systems

Picture courtesy: Bhargava and Forsyth (2019), https://www.thestranger.com/, Xiaolin Wu and Xi Zhang, “Automated Inference on Criminality Using Face Images”
 22
Machine Learning Tools
 Scikit-Learn
 Pytorch
 Google Colab
 Jupyter Notebook
 Tensorflow
 Google Cloud AutoML
 Numpy
 MatPlotLib
 Open CV
 Microsoft Azure Machine Learning
 Amazon Machine Learning
 23
Next Class
 Various Flavors of ML problems

 Data and features

 Basic mathematical operations on data and features