Gain insights into linear algebra essentials for data science, focusing on vectors, matrices, and tensors. Explore practical Python applications, engaging visuals, real datasets, and hands-on projects.

linearalgebra.tar.gz

Code

visualizing-planes-3D

line-graph

convex-combinations-animation

intersecting planes

Linear algebra is a fundamental pillar of data science. In advanced models in data science, like neural networks, the inputs and transformations are based upon vectors, matrices, and tensors which require a reasonable understanding of linear algebra to get the desired results. It is elegant and the most applied mathematics under the umbrella of data science.

This course teaches linear algebra with a focus on data science. This course encompasses several engaging illustrations, including static images and animations. Furthermore, this course presents mathematical modeling through programming in Python. This course contains several executable coding playgrounds on real data sets and a final project with practical applications.

Aside from theoretical implementations, the modern-day world needs its daunting calculations, trajectory mapping, and distance manipulation, all of which linear algebra provides. By the end of this course, you’ll have a working knowledge of all the necessary teachings in linear algebra.

Linear Algebra for Data Science Using Python

# What is linear independence?
Assume that we have a beautiful blue toy car with a plastic body and rubber tires. We may say that the toy car is constructed using some ratio of plastic to rubber. In other words, the material of the toy car is a linear combination of these. Moreover, we can say that the toy car is dependent on plastic and rubber, whereas it’s independent of wood because wood isn’t used in its construction.

The same concept can be extended to mathematical objects. If a mathematical object is made using a linear combination of others, then we call such an object linearly dependent on those objects. Otherwise, it’s independent of objects with linear combinations that aren’t used in its construction. Let’s write this linear independence concept formally.

## Formal definition
An object, $\bold{z}$, is linearly independent of objects $\bold{x_1},\bold{x_2},\cdots,\bold{x_n}$ if and only if no choice of scalars $w_1,w_2,\cdots,w_n$ exists such that $\bold{z} = w_1\bold{x_1}+w_2\bold{x_2}+\cdots+w_n\bold{x_n}$.

## Example: Axis of $xyz$ space

One way to represent the $x,y$, and $z$ axes in the standard $xyz$ space is $\begin{bmatrix}
1 \\
0  \\
0 
\end{bmatrix},\begin{bmatrix}
0 \\
1  \\
0 
\end{bmatrix}$, and $\begin{bmatrix}
0 \\
0  \\
1 
\end{bmatrix}$, respectively. There’s no way to generate any one axis from the others linearly! These are all linearly independent of each other. Any linear combination of the $x$ and $y$ axes stays in the $xy$ plane. This is true for any other pair of axes.

# Test for linear independence
Given an object $\bold{z}$, how can we test if it's linearly independent of other given objects, say $\bold{x_1},\bold{x_2},...,\bold{x_n}$? Typically, there are infinitely many possible linear combinations of objects. Therefore, it isn't possible to check all of them one by one. We'll learn a formal method of testing the linear independence of an object after the lessons about matrices. In the meantime, there are some useful tricks  to quickly check if a set of objects are linearly independent. Although these tricks can generally be applied to any mathematical object, we'll focus on vectors for simplicity. Formally, a collection of vectors
$\bold{x_1},\bold{x_2},\cdots,\bold{x_n}$  is called linearly dependent if there exists a collection of scalars $w_1,w_2,\cdots ,w_n$, such that:
$$w_1\bold{x_1}+w_2\bold{x_2}+\cdots+w_n\bold{x_n}=\bold{0}$$
>**Note:** The $\bold{0}$ on the right-hand side of the equation above is a zero vector.
><br> At least one scalar from $w_1,w_2,\cdots,w_n$ must be non-zero.

## Set of a single vector
A single non-zero vector, $\bold{x}$, is always linearly independent. This is because it’s impossible to find any non-zero scalar $w$, such that $w\bold{x}=\bold{0}$. 

## Set containing a zero vector
A set containing a zero vector will always be linearly dependent. We can always create a linear combination of that set, satisfying the above definition. For example, assume we have a set of three vectors, $\{\bold{x_1}, \bold{x_2}, \bold{x_3}\}$, in which $\bold{x_3}$  is a zero vector. We can then fulfill the definition requirements above with: $0\bold{x_1}+ 0\bold{x_2}+99\bold{x_3} = \bold{0}$.

## Set of too many vectors
A set where each vector has $n$ elements but contains more than $n$ vectors will always be linearly dependent. For example, this set: $\bigg\{\begin{bmatrix}0\\1\end{bmatrix}, \begin{bmatrix}1\\-1\end{bmatrix}, \begin{bmatrix}13\\7\end{bmatrix}\bigg\}$ is linearly dependent because each vector has two elements, but the set has more than two vectors. We’ll explain the reason behind this in a future lesson.

## Set of two vectors
If a set has only two vectors, then it will be linearly dependent if and only if one vector is a scalar multiple of the other. For example, the set $\Bigg\{\bold{x_1}=\begin{bmatrix}3\\7\\0\end{bmatrix}, \bold{x_2}=\begin{bmatrix}6\\14\\0\end{bmatrix} \Bigg\}$ is linearly dependent as the second vector is twice the first vector, i.e., $\bold{x_2}=2\bold{x_1}$. We can fulfill the definition above by creating a linear combination, namely  $\bold{x_2}-2\bold{x_1}=\bold{0}$.

  

Learn about linear independence and methods to quickly check if a set is linearly independent.

Introduction

Linearity

Matrices

Solving Linear Systems

Singularity

Linear Regression and Least Squares

Vector Space

Vector Spaces of a Matrix

Singular Value Decomposition: SVD

Learning to Find Discriminative Null Space for Face Recognition

Linear Independence

What is linear independence?

Formal definition