Gain insights into linear algebra essentials for data science, focusing on vectors, matrices, and tensors. Explore practical Python applications, engaging visuals, real datasets, and hands-on projects.

linearalgebra.tar.gz

Code

visualizing-planes-3D

line-graph

convex-combinations-animation

intersecting planes

Linear algebra is a fundamental pillar of data science. In advanced models in data science, like neural networks, the inputs and transformations are based upon vectors, matrices, and tensors which require a reasonable understanding of linear algebra to get the desired results. It is elegant and the most applied mathematics under the umbrella of data science.

This course teaches linear algebra with a focus on data science. This course encompasses several engaging illustrations, including static images and animations. Furthermore, this course presents mathematical modeling through programming in Python. This course contains several executable coding playgrounds on real data sets and a final project with practical applications.

Aside from theoretical implementations, the modern-day world needs its daunting calculations, trajectory mapping, and distance manipulation, all of which linear algebra provides. By the end of this course, you’ll have a working knowledge of all the necessary teachings in linear algebra.

Linear Algebra for Data Science Using Python

# Squared error
Squared distance is also known as squared error. Consider a linear equation in  $w_i$'s: 
$$w_1a_1+w_2a_2+...+w_na_n=b$$

The squared error (squared distance) on a given point, $(\hat w_1,\hat w_2,...,\hat w_n)$, is defined as:

$$SE(\hat w_1,\hat w_2,...,\hat w_n)=(\hat w_1a_1+\hat w_2a_2+...+\hat w_na_n-b)^2$$

>**Note:** In the case of $w=\hat{w}$, the sum of squared errors=0. This implies that we’re able to find an *exact* solution.

# Sum of squared errors

Consider a linear system with $m$ equations and $n$ unknowns and the corresponding squared errors on a point, $(\hat w_1,\hat w_2,...,\hat w_n)$:



| **Linear System**  | **Sum of Squared Distances**  |
| :- | :- |
|$(1):w_1a_{11}+w_2a_{12}+...+w_na_{1n}=b_1$$(2):w_1a_{21}+w_2a_{22}+...+w_na_{2n}=b_2$$\;\ \;\ \;\ \;\ \;\ \;\  \;\ \;\        \vdots$$(m):w_1a_{m1}+w_2a_{m2}+...+w_na_{mn}=b_m$| $SE_1=(\hat w_1a_{11}+\hat w_2a_{12}+...+\hat w_na_{1n}-b_1)^2$$SE_2=(\hat w_1a_{21}+\hat w_2a_{22}+...+\hat w_na_{2n}-b_2)^2$$\;\ \;\ \;\ \;\ \;\ \;\  \;\ \vdots$$SE_m=(\hat w_1a_{m1}+\hat w_2a_{m2}+...+\hat w_na_{mn}-b_m)^2$|






>**Note:** For conciseness, we've written $SE_i(\hat w_1,\hat w_2,...,\hat w_n)$ as $SE_i$.


Now, the sum of squared errors (sum of squared distances) is simply:


$$SSE(\hat w_1,\hat w_2,...,\hat w_n)=\sum_{i=1}^{m}SE_i(\hat w_1,\hat w_2,...,\hat w_n)$$



>**Note:** $SSE$ is a function of unknowns $(\hat w_1,\hat w_2,...,\hat w_n)$.



## Vectorized notation

Consider the vectors:
$$
\bold{w} = \begin{bmatrix}w_1 \\ w_2 \\
\vdots \\ w_n\end{bmatrix},
\bold{b} = \begin{bmatrix}b_1 \\ b_2 \\ \vdots \\b_n \end{bmatrix},
\bold{a_i} = \begin{bmatrix}a_{i1} \\ a_{i2} \\ \vdots \\ a_{in} \end{bmatrix}  \text{, and} \quad  \bold{\hat w}=\begin{bmatrix}\hat w_1\\\hat w_2 \\\vdots \\\hat w_n\end{bmatrix} $$


then the squared error for the $ith$ equation is

$$SE_i(\bold{\hat w})=(\bold{\hat w}^T\bold{a_i}-b_i)^2=(\bold{ a_i}^T\bold{\hat w}-b_i)^2$$

Now, if we represent the augmented matrix, 

$A$ as $A = \begin{bmatrix}\bold{a_1^T}\\\bold{a_2^T}\\\vdots\\\bold{a_m^T}\end{bmatrix}$, and the *algebraic distance* of all equations as vector $\bold{d} = \begin{bmatrix}d_1\\d_2\\\vdots\\d_m\end{bmatrix}$, where $\bold{d}=A\bold{\hat w}-\bold{b}$, it is easy to comprehend that:
$$SE_i(\bold{\hat w})=d_i^2$$




Because, by definition, SSE is the sum of all $SE_i$, we can write: 
$$SSE(\bold{\hat w})=\sum_{i=1}^{m} d_i^2=\bold{d}^T\bold{d}=(A\bold{\hat w}-\bold{b})^T(A\bold{\hat w}-\bold{b})$$


## Example
Recall the following inconsistent linear system from the previous lesson:

$$ \begin{align}
    w_1-2w_2&=9 \\
    3w_1+w_2&=5 \\
    w_1+w_2&=8
\end{align} $$

Learn about approximating the solution of an inconsistent linear system through least squares.

Introduction

Linearity

Matrices

Solving Linear Systems

Singularity

Linear Regression and Least Squares

Vector Space

Vector Spaces of a Matrix

Singular Value Decomposition: SVD

Learning to Find Discriminative Null Space for Face Recognition

Least Squared Error Solution

Squared error

Sum of squared errors