Function approximation

Approximating a function means estimating the values of its parameters. Consider the SSE function we discussed in the previous lesson.

SSE(\bold{\hat w})=(A\bold{\hat w}-\bold{b})^T(A\bold{\hat w}-\bold{b})

Approximating SSE means estimating the vector, $\bold w$ , that nearly satisfies the linear system, also called the linear least squared error solution.

Formal definition

Consider a data set, $D=\{(\bold{x_1},\bold{y_1}),(\bold{x_2},\bold{y_2}),...,(\bold{x_n},\bold{y_n})\}$ , where each entry is a pair, $\bold{x_i}$ and $\bold{y_i}$ , of objects (scalars, vectors, matrices, and so on). Function approximation seeks a function, $f_\bold{w}$ , such that:

f_\bold{w}(\bold{x_i})\approx\bold{y_i}

Example

Let $D=\{(4,1),(-3,9)\}$ . The function $f_\bold{w}$ represents a line passing through the two data points in the $xy$ plane. However, infinitely many non-linear curves pass through the same data points. A few of these curves are shown in the figure below.

In the figure to the right, the green points are from the data set. A line, in the color red, and two different curves, of colors black and blue, are passing through the data points. As we can see, all the functions (red line, blue curve, and black curve) approximate the data rather exactly. In this case, we can find the exact function(s) that fit data. When an exact fit is hard to estimate, we may rely on an approximate fit, that is, a curve which is near to the data points.

Curve fitting in data points in green colors

Press + to interact

Introduction

Linearity

Matrices

Solving Linear Systems

Singularity

Linear Regression and Least Squares

Vector Space

Vector Spaces of a Matrix

Singular Value Decomposition: SVD

Learning to Find Discriminative Null Space for Face Recognition

Linear Regression

Function approximation

Formal definition

Example

Regression vs. classification