Linear Regression with multiple variables(다변량선형회귀)

Multiple features

Machine Learning

Andrew Ng

Size (feet2) Price ($1000)

2104 4601416 2321534 315852 178… …

다수의특징들 (변수: variables).

Andrew Ng

Size (feet2) Number of bedrooms

Number of floors

Age of home (years)

Price ($1000)

2104 5 1 45 4601416 3 2 40 2321534 3 2 30 315852 2 1 36 178… … … … …

Multiple features (variables).

Notation:

= number of features

= input (features) of training example.

= value of feature in training example.

Andrew Ng

Hypothesis:

Previously:

Andrew Ng

For convenience of notation, define .

Multivariate linear regression.

Linear Regression with multiple variables

Gradient descent for multiple variables

Machine Learning

Andrew Ng

Hypothesis:

Cost function:

Parameters:

(simultaneously update for every )

Repeat

Gradient descent:

Andrew Ng

(simultaneously update )

Gradient Descent

Repeat

Previously (n=1):

New algorithm :

Repeat

(simultaneously update for )

Linear Regression with multiple variables

Gradient descent in practice I: Feature Scaling

Machine Learning

Andrew Ng

E.g. = size (0-2000 feet2)

= number of bedrooms (1-5)

Feature ScalingIdea: 특징들이유사한스케일이되게하는것.

size (feet2)

number of bedrooms

Andrew Ng

특징크기조정(Feature Scaling)

모든특징들이대략적으로 구간이되게 ..

Andrew Ng

를 로교체 <= 특징들의평균이대략적으로 0 이되도록

(Do not apply to ).

평균정규화(Mean normalization)

E.g.

Linear Regression with multiple variables

Gradient descent in practice II: Learning rate

Machine Learning

Andrew Ng

Gradient descent

- “Debugging”: 어떻게경사하강을제대로

작동되게 할수있나

- 학습률 를어떻게선택하는가? .

Andrew Ng

Example automatic

convergence test:

Declare convergence if

decreases by less than

in one iteration.0 100 200 300 400

No. of iterations

Making sure gradient descent is working correctly.

Andrew Ng

Making sure gradient descent is working correctly.

Gradient descent not working.

Use smaller .

No. of iterations

No. of iterations No. of iterations

- For sufficiently small , should decrease on every iteration.- But if is too small, gradient descent can be slow to converge.

Andrew Ng

Summary:

- 가너무작으면: 천천히수렴.

- 가너무크면 : 가반복때마다감소하지

않을수있고; 수렴안할수도있다.

To choose , try

Linear Regression with multiple variables

Features and polynomial regression

Machine Learning

Andrew Ng

집값예측(Housing prices prediction)

Andrew Ng

다항식회귀(Polynomial regression)

Price(y)

Size (x)

Andrew Ng

특징선택(Choice of features)

Price(y)

Size (x)

Linear Regression with multiple variables

정규방정식

(Normal equation)

Machine Learning

Andrew Ng

Gradient Descent

Normal equation: 를해석적으로(analytically)

푸는방법.

Andrew Ng

Intuition: If 1D

Solve for

(for every )

Andrew Ng

Size (feet2) Number of bedrooms

Number of floors

Age of home (years)

Price ($1000)

1 2104 5 1 45 4601 1416 3 2 40 2321 1534 3 2 30 3151 852 2 1 36 178

Size (feet2) Number of bedrooms

Number of floors

Age of home (years)

Price ($1000)

2104 5 1 45 4601416 3 2 40 2321534 3 2 30 315852 2 1 36 178

Examples:

Andrew Ng

examples ; features.

E.g. If

Andrew Ng

is inverse of matrix .

Octave: pinv(X’*X)*X’*y

Andrew Ng

training examples, features.

Gradient Descent Normal Equation

• No need to choose .• Don’t need to iterate.

• Need to choose . • Needs many iterations.• Works well even

when is large.

• Need to compute

• Slow if is very large.

Linear Regression with multiple variables

Normal equation and non-invertibility(optional)

Machine Learning

Andrew Ng

Normal equation

- What if is non-invertible? (singular/ degenerate)

- Octave: pinv(X’*X)*X’*y

Andrew Ng

What if is non-invertible?

• Redundant features (linearly dependent).E.g. size in feet2

size in m2

• Too many features (e.g. ).

- Delete some features, or use regularization.