linear regression with multiple variablesjun.hansung.ac.kr/ml/docs-slides-lecture4-kr.pdf · 2016....
TRANSCRIPT
Linear Regression with multiple variables(다변량선형회귀)
Multiple features
Machine Learning
Andrew Ng
Size (feet2) Price ($1000)
2104 4601416 2321534 315852 178… …
다수의특징들 (변수: variables).
Andrew Ng
Size (feet2) Number of bedrooms
Number of floors
Age of home (years)
Price ($1000)
2104 5 1 45 4601416 3 2 40 2321534 3 2 30 315852 2 1 36 178… … … … …
Multiple features (variables).
Notation:
= number of features
= input (features) of training example.
= value of feature in training example.
Andrew Ng
Hypothesis:
Previously:
Andrew Ng
For convenience of notation, define .
Multivariate linear regression.
Linear Regression with multiple variables
Gradient descent for multiple variables
Machine Learning
Andrew Ng
Hypothesis:
Cost function:
Parameters:
(simultaneously update for every )
Repeat
Gradient descent:
Andrew Ng
(simultaneously update )
Gradient Descent
Repeat
Previously (n=1):
New algorithm :
Repeat
(simultaneously update for )
Linear Regression with multiple variables
Gradient descent in practice I: Feature Scaling
Machine Learning
Andrew Ng
E.g. = size (0-2000 feet2)
= number of bedrooms (1-5)
Feature ScalingIdea: 특징들이유사한스케일이되게하는것.
size (feet2)
number of bedrooms
Andrew Ng
특징크기조정(Feature Scaling)
모든특징들이대략적으로 구간이되게 ..
Andrew Ng
를 로교체 <= 특징들의평균이대략적으로 0 이되도록
(Do not apply to ).
평균정규화(Mean normalization)
E.g.
Linear Regression with multiple variables
Gradient descent in practice II: Learning rate
Machine Learning
Andrew Ng
Gradient descent
- “Debugging”: 어떻게경사하강을제대로
작동되게 할수있나
- 학습률 를어떻게선택하는가? .
Andrew Ng
Example automatic
convergence test:
Declare convergence if
decreases by less than
in one iteration.0 100 200 300 400
No. of iterations
Making sure gradient descent is working correctly.
Andrew Ng
Making sure gradient descent is working correctly.
Gradient descent not working.
Use smaller .
No. of iterations
No. of iterations No. of iterations
- For sufficiently small , should decrease on every iteration.- But if is too small, gradient descent can be slow to converge.
Andrew Ng
Summary:
- 가너무작으면: 천천히수렴.
- 가너무크면 : 가반복때마다감소하지
않을수있고; 수렴안할수도있다.
To choose , try
Linear Regression with multiple variables
Features and polynomial regression
Machine Learning
Andrew Ng
집값예측(Housing prices prediction)
Andrew Ng
다항식회귀(Polynomial regression)
Price(y)
Size (x)
Andrew Ng
특징선택(Choice of features)
Price(y)
Size (x)
Linear Regression with multiple variables
정규방정식
(Normal equation)
Machine Learning
Andrew Ng
Gradient Descent
Normal equation: 를해석적으로(analytically)
푸는방법.
Andrew Ng
Intuition: If 1D
Solve for
(for every )
Andrew Ng
Size (feet2) Number of bedrooms
Number of floors
Age of home (years)
Price ($1000)
1 2104 5 1 45 4601 1416 3 2 40 2321 1534 3 2 30 3151 852 2 1 36 178
Size (feet2) Number of bedrooms
Number of floors
Age of home (years)
Price ($1000)
2104 5 1 45 4601416 3 2 40 2321534 3 2 30 315852 2 1 36 178
Examples:
Andrew Ng
examples ; features.
E.g. If
Andrew Ng
is inverse of matrix .
Octave: pinv(X’*X)*X’*y
Andrew Ng
training examples, features.
Gradient Descent Normal Equation
• No need to choose .• Don’t need to iterate.
• Need to choose . • Needs many iterations.• Works well even
when is large.
• Need to compute
• Slow if is very large.
Linear Regression with multiple variables
Normal equation and non-invertibility(optional)
Machine Learning
Andrew Ng
Normal equation
- What if is non-invertible? (singular/ degenerate)
- Octave: pinv(X’*X)*X’*y
Andrew Ng
What if is non-invertible?
• Redundant features (linearly dependent).E.g. size in feet2
size in m2
• Too many features (e.g. ).
- Delete some features, or use regularization.