Linear regression
with one variable
Supervised Learning
Regression Problem: Predict real-valued output
Andrew Ng
Supervised Learning
The computer is presented with example inputs and
their desired outputs, and the goal is to learn a general
rule that maps inputs to outputs.
Andrew Ng
Supervised Learning: Regression
• Goal: Determine the function, which maps x to y
• Function: Approximated using the dataset
• The machine learns, for what value of x, what value
of y is usually obtained
• Formulated as a function
• Any unseen x as input provides an expected y
Andrew Ng
Training set of housing Size in feet2 (x) Price ($) in 1000's (y)
prices
(Portland, OR) 2104 460
Linear regression with one variable.
1416 232
Univariate linear regression. 1534 315 m
One variable 852 178
… …
Notation:
m = Number of training examples
x’s = “input” variable / features
x(1) = 2104
y’s = “output” variable / “target” variable
x(2) = 1416
(x, y) – one training example y(1) = 460
(x(i), y(i)) – ith trainingg example
Andrew Ng
Andrew Ng
Andrew Ng
Andrew Ng
How do we represent h ?
Training Set
Learning Algorithm
Size of h Estimated
house price
Degree 7 polynomial
x hypothesis y Linear hypothesis
hypothesis
h maps from x’s to y’s
Restriction bias: Consider only
linear functions
Andrew Ng
Size in feet2 (x) Price ($) in 1000's (y)
Training Set
2104 460
1416 232
1534 315
852 178
… …
Hypothesis:
‘s: Parameters
How to choose ‘s ?
Andrew Ng
3 3 3
h(x) = 1.5 + 0·x h(x) = 0.5·x
2 2 2
1 1 1
h(x) = 1 + 0.5·x
0 0 0
0 1 2 3 0 1 2 3 0 1 2 3
Andrew Ng
(x(i), y(i))
1 𝑚 (i)) −y(i) )2
y minimize 𝑖=1 (hΘ(x
Θ0, Θ1 Θ0 Θ1 2𝑚
h(x(i)) = Θ0+ Θ1x(i)
x
1 𝑚 (i)) −y(i) )2
J(Θ0, Θ1) = 𝑖=1 (hΘ(x
2𝑚
Idea: Choose so that
Minimize J(Θ0, Θ1) : Cost Function
is close to for our Θ0 Θ1
training examples Squared error function
Andrew Ng
Linear regression
with one variable
Cost function
intuition I
Machine Learning
Andrew Ng
Simplified
Hypothesis:
Parameters:
h(x) Θ0 = 0
h(x)
Cost Function:
Goal:
Andrew Ng
(for fixed , this is a function of x) (function of the parameter )
3 3
2 2
y
1 1
0 0
0 1 x 2 3 -0.5 0 0.5 1 1.5 2 2.5
1 𝑚 (i)) − y(i))2
J(Θ1) = 𝑖=1 (hΘ(x
2𝑚
1 𝑚 (i)) − y(i)) 2
= 𝑖=1 (Θ1x 𝐽 1 =0
2𝑚
1
= 02 + 02 + 02
2𝑚 Andrew Ng
(for fixed , this is a function of x) (function of the parameter )
3 3
2 2
y y(i)
1 1
hΘ(x(i))
0 0
0 1 2 3 -0.5 0 0.5 1 1.5 2 2.5
x
1 3
J(0.5) = 𝑖=1 [(0.5−1)2 +(1−2)2+(1.5−3)2]
2∙3
1
= ∙(3.5) = 0.58
6
Andrew Ng
(for fixed , this is a function of x) (function of the parameter )
3 3
2 2
y
1 1
0 0
0 1 2 3 -0.5 0 0.5 1 1.5 2 2.5
x
1 3 2
J(0) = 𝑖=1 [1 +22+32]
2∙3
1
= 6
∙ 14 = 2.3
Andrew Ng
Linear regression
with one variable
Cost function
intuition II
Machine Learning
Andrew Ng
Hypothesis:
Parameters:
Cost Function:
Goal:
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
500
400
Price ($) 300
in 1000’s
200
Θ0 = 50
100
Θ1 = 0.06
0
0 1000 2000 3000
Size in feet2 (x)
Andrew Ng
Contour plots
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
Is the slope positive or negative What is the value of
Do you agree?
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
h(x) = 360 + 0·x
Θ0 = 360
Θ1 = 0
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
Is the slope positive or negative What is the value of
Do you agree?
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
Is the slope positive or negative What is the value of
Do you agree?
Andrew Ng
Linear regression
with one variable
Gradient
Machine Learning
descent
Andrew Ng
Have some function
Want
Outline:
• Start with some
• Keep changing to reduce
until we hopefully end up at a minimum
Andrew Ng
J(0,1)
1
0
Andrew Ng
J(0,1)
1
0
Andrew Ng
Linear regression
with one variable
Gradient descent for
linear regression
Machine Learning
Andrew Ng
J(0,1)
1
0
Andrew Ng
Convex function
Bowl-shaped
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
Andrew Ng
