Gain insights into linear algebra essentials for data science, focusing on vectors, matrices, and tensors. Explore practical Python applications, engaging visuals, real datasets, and hands-on projects.

linearalgebra.tar.gz

Code

visualizing-planes-3D

line-graph

convex-combinations-animation

intersecting planes

Linear algebra is a fundamental pillar of data science. In advanced models in data science, like neural networks, the inputs and transformations are based upon vectors, matrices, and tensors which require a reasonable understanding of linear algebra to get the desired results. It is elegant and the most applied mathematics under the umbrella of data science.

This course teaches linear algebra with a focus on data science. This course encompasses several engaging illustrations, including static images and animations. Furthermore, this course presents mathematical modeling through programming in Python. This course contains several executable coding playgrounds on real data sets and a final project with practical applications.

Aside from theoretical implementations, the modern-day world needs its daunting calculations, trajectory mapping, and distance manipulation, all of which linear algebra provides. By the end of this course, you’ll have a working knowledge of all the necessary teachings in linear algebra.

Linear Algebra for Data Science Using Python

## Linear dependence
We defined linear dependence as a collection with linearly dependent vectors if, given a collection of vectors  $\bold x_1, \bold x_2, \cdots, \bold x_n$, there exists a combination of scalars $w_1,w_2,\cdots,w_n$ with at least one nonzero value such that
$$w_1\bold{x_1}+w_2\bold{x_2}+...+w_n\bold{x_n}=\bold{0}$$

There could be infinitely many possible linear combinations, and it's not possible to check them one by one. We’ve already discussed a few special cases and tricks to check for linear dependence in a combination. This lesson will use elimination as a formal method to solve the vector equation and find the possible combinations.
 
## Procedure
Our goal is to find the combination of scalars $w_1,w_2,\cdots,w_n$, such that
$$w_1\bold{x_1}+w_2\bold{x_2}+...+w_n\bold{x_n}=\bold{0}$$
Expanding the vectors, we get
$$
w_1 \begin{bmatrix} x_{11} \\ x_{12} \\ \vdots \\ x_{1m} \end{bmatrix}
+
w_2 \begin{bmatrix} x_{21} \\ x_{22} \\ \vdots \\ x_{2m} \end{bmatrix}
+
\cdots
+
w_n \begin{bmatrix} x_{n1} \\ x_{n2} \\ \vdots \\ x_{1m} \end{bmatrix}
=
\begin{bmatrix} 0 \\ 0 \\ \vdots \\ 0 \end{bmatrix}
$$
The equation can be converted to a linear system of $m$ equations in $n$ variables. The augmented matrix representation of the system is as follows:
$$\left( \begin{array}{cccc|c}
x_{11} & x_{21} & \cdots & x_{n1} & 0 \\
x_{12} & x_{22} & \cdots & x_{n2} & 0 \\
\vdots & \vdots & \ddots & \vdots & \vdots\\
x_{1m} & x_{2m} & \cdots & x_{nm} & 0 \\
\end{array}\right)$$

Because the vector $\bold b$ is a zero vector, the system is of the form $A\bold{w}=\bold{0}$ and is always consistent. The two possible solution sets, “only trivial solution” and “infinite solutions,” provide evidence for linear independence and linear dependence, respectively.


### Only trivial solution
Suppose the $rref$ of $A$ for the system $A\bold w=\bold 0$ results in an identity matrix (that is, it has no free variable). In such a case, the only possible solution is the trivial solution, $\bold w=\bold 0$. This means that the only combination of scalars satisfying the vector equation is all zeros, and the vectors included in the collection are independent of each other.

**Example**

Consider the following set of vectors:
$$
\bold {x_1} = \begin{bmatrix} 5 \\ 2 \\  4 \end{bmatrix},
\bold {x_2} = \begin{bmatrix} 0 \\ 1 \\  1 \end{bmatrix},
\bold {x_3} = \begin{bmatrix} 10 \\ 9 \\  15 \end{bmatrix}
$$
Writing the vector equation, we get
$$
w_1 \begin{bmatrix} 5 \\ 2 \\ 4 \end{bmatrix} +
w_2 \begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix} +
w_3 \begin{bmatrix} 10 \\ 9 \\ 15 \end{bmatrix}
= \begin{bmatrix} 0 \\ 0  \\ 0  \end{bmatrix}
$$
Our next step is to write an augmented matrix and apply elimination.

$$\left( \begin{array}{ccc|c}
5 & 0 & 10 & 0 \\
2 & 1 & 9 & 0 \\
4 & 1 & 15 & 0 \\
\end{array}\right)$$

# Linear dependence
We defined linear dependence as a collection with linearly dependent vectors if, given a collection of vectors  $\bold x_1, \bold x_2, \cdots, \bold x_n$, there exists a combination of scalars $w_1,w_2,\cdots,w_n$ with at least one nonzero value such that
$$w_1\bold{x_1}+w_2\bold{x_2}+...+w_n\bold{x_n}=\bold{0}$$

There could be infinitely many possible linear combinations, and it's not possible to check them one by one. We’ve already discussed a few special cases and tricks to check for linear dependence in a combination. This lesson will use elimination as a formal method to solve the vector equation and find the possible combinations.
 
# Procedure
Our goal is to find the combination of scalars $w_1,w_2,\cdots,w_n$, such that
$$w_1\bold{x_1}+w_2\bold{x_2}+...+w_n\bold{x_n}=\bold{0}$$
Expanding the vectors, we get
$$
w_1 \begin{bmatrix} x_{11} \\ x_{12} \\ \vdots \\ x_{1m} \end{bmatrix}
+
w_2 \begin{bmatrix} x_{21} \\ x_{22} \\ \vdots \\ x_{2m} \end{bmatrix}
+
\cdots
+
w_n \begin{bmatrix} x_{n1} \\ x_{n2} \\ \vdots \\ x_{1m} \end{bmatrix}
=
\begin{bmatrix} 0 \\ 0 \\ \vdots \\ 0 \end{bmatrix}
$$
The equation can be converted to a linear system of $m$ equations in $n$ variables. The augmented matrix representation of the system is as follows:
$$\left( \begin{array}{cccc|c}
x_{11} & x_{21} & \cdots & x_{n1} & 0 \\
x_{12} & x_{22} & \cdots & x_{n2} & 0 \\
\vdots & \vdots & \ddots & \vdots & \vdots\\
x_{1m} & x_{2m} & \cdots & x_{nm} & 0 \\
\end{array}\right)$$

Because the vector $\bold b$ is a zero vector, the system is of the form $A\bold{w}=\bold{0}$ and is always consistent. The two possible solution sets, “only trivial solution” and “infinite solutions,” provide evidence for linear independence and linear dependence, respectively.


## Only trivial solution
Suppose the $rref$ of $A$ for the system $A\bold w=\bold 0$ results in an identity matrix (that is, it has no free variable). In such a case, the only possible solution is the trivial solution, $\bold w=\bold 0$. This means that the only combination of scalars satisfying the vector equation is all zeros, and the vectors included in the collection are independent of each other.

**Example**

Consider the following set of vectors:
$$
\bold {x_1} = \begin{bmatrix} 5 \\ 2 \\  4 \end{bmatrix},
\bold {x_2} = \begin{bmatrix} 0 \\ 1 \\  1 \end{bmatrix},
\bold {x_3} = \begin{bmatrix} 10 \\ 9 \\  15 \end{bmatrix}
$$
Writing the vector equation, we get
$$
w_1 \begin{bmatrix} 5 \\ 2 \\ 4 \end{bmatrix} +
w_2 \begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix} +
w_3 \begin{bmatrix} 10 \\ 9 \\ 15 \end{bmatrix}
= \begin{bmatrix} 0 \\ 0  \\ 0  \end{bmatrix}
$$
Our next step is to write an augmented matrix and apply elimination.

$$\left( \begin{array}{ccc|c}
5 & 0 & 10 & 0 \\
2 & 1 & 9 & 0 \\
4 & 1 & 15 & 0 \\
\end{array}\right)$$

Learn to verify if given vectors are linearly independent, and if not, find the relationship between the dependent vectors.

Introduction

Linearity

Matrices

Solving Linear Systems

Singularity

Linear Regression and Least Squares

Vector Space

Vector Spaces of a Matrix

Singular Value Decomposition: SVD

Learning to Find Discriminative Null Space for Face Recognition

Verifying Linear Independence

Linear dependence

Procedure