Nonlinear Models

What we are really doing in statistical learning is to find a approximator for the data.

Suppose we have a model for some data X, which describes the data as

Y = F(X),

where F(X) is some function of data and Y is the prediction. In principle, it can be any function since data can be very complicated. However, Taylor expansion tells us that

F(X) = \hat \beta_0  + \hat \beta_1 x_1 +\hat \beta_2 x_2 +\cdots.

Think about linear regression. It’s just an approximator that neglects the higher orders. In principle we could put in higher orders for the data. These models with higher orders are called polynomial regression.

In general, we could think of other forms, such as fourier series, wavelets, step functions, etc.

Polynomial Regression

Tails in Polynomial Regression

Polynomials are wiggly at the tails. We usually discard them.

Step Functions

© 2018, Lei Ma| GitHub| Statistical Mechanics Notebook | Index | Page Source| changelog| Created with Sphinx