## Logistic Regression(逻辑回归)

Don’t be confused by the name “Logistic Regression”, it is named that way for historical reasons and is actually an approach to classification problems, not regression problems. 别被名字误导了，实际是解决分类问题的。

### Binary Classification

**Hypothesis** should satisfy:

“**Sigmoid Function**,” also called the “**Logistic Function**”:

will give us the **probability** that our output is 1 or 0.

二分类满足：

### Decision Boundary

根据**Sigmoid Function** 可以推出：

### Cost Function

Cost function for logistic regression looks like:

The more our hypothesis is off from y, the larger the cost function output. If our hypothesis is equal to y, then our cost is 0:

### Simplified Cost Function and Gradient Descent

#### Simplified Cost Function

We can compress our cost function’s two conditional cases into one case:

A **vectorized** implementation is:

#### Gradient Descent

We can work out the derivative part using calculus to get:

A **vectorized** implementation is:

#### Partial derivative of J(θ)

The **vectorized** version:

### Advanced Optimization

**Conjugate gradient**
**BFGS**
**L-BFGS**

### Multiclass Classification: One-vs-all

## Regularization

Regularization is designed to address the problem of **overfitting**.
There are two main options to address the issue of overfitting:

**1. Reduce the number of features:**

- Manually select which features to keep.
- Use a model selection algorithm.

**2. Regularization**

- Keep all the features, but reduce the parameters.
- Regularization works well when we have a lot of slightly useful features.