Machine Learning_Week3

Logistic Regression(逻辑回归)

Don’t be confused by the name “Logistic Regression”, it is named that way for historical reasons and is actually an approach to classification problems, not regression problems. 别被名字误导了,实际是解决分类问题的。

Binary Classification

Hypothesis should satisfy:

Sigmoid Function,” also called the “Logistic Function”:

will give us the probability that our output is 1 or 0.

二分类满足:

Decision Boundary

根据Sigmoid Function 可以推出:

Cost Function

Cost function for logistic regression looks like:

The more our hypothesis is off from y, the larger the cost function output. If our hypothesis is equal to y, then our cost is 0:

Simplified Cost Function and Gradient Descent

Simplified Cost Function

We can compress our cost function’s two conditional cases into one case:

A vectorized implementation is:

Gradient Descent

We can work out the derivative part using calculus to get:

A vectorized implementation is:

Partial derivative of J(θ)

The vectorized version:

Advanced Optimization

Conjugate gradient BFGS L-BFGS

Multiclass Classification: One-vs-all

Regularization

Regularization is designed to address the problem of overfitting. There are two main options to address the issue of overfitting:

1. Reduce the number of features:

  • Manually select which features to keep.
  • Use a model selection algorithm.

2. Regularization

  • Keep all the features, but reduce the parameters.
  • Regularization works well when we have a lot of slightly useful features.

Cost Function

Regularized Linear Regression

Gradient Descent

Normal Equation

Regularized Logistic Regression

Cost Function

Gradient Descent

Reference

Machine Learning by Stanford University

Yuehua(刘跃华) wechat