Matrix Multiplication

One of the most commonly used operations in data science is matrix multiplication. Because of its frequency in predictive models (including deep neural networks), a lot of attention has been devoted to making matrix multiplication more efficient in recent years. Today, we even have special hardware for fast matrix multiplication. In this lesson, we’ll learn about matrix multiplication and different ways to do it.

The dimensions rule

Not every pair of matrices can be multiplied together. Two matrices are multipliable if the number of columns, $n$, in the first matrix equals the number of rows in the second matrix.

In other words, we can multiply any two matrices having equal inner dimensions, and the result would be a new matrix with an order equaling the outer dimensions:

Dot product

The simplest way to understand matrix multiplication is through dot product. The dot product of two vectors is simply the sum of products of corresponding elements. Consider the vectors $\bold{a}=(1,4,7)$ and $\bold{b}=(3,1,2)$. The dot product of $\bold{a}$ and $\bold{b}$ is $(1*3+4*1+7*2)=21$. A general representation of a dot product is

$$c = \sum_{k=1}^{n}a_{k}*b_{k}$$

The same dot product can be used to represent matrix multiplication. Consider the matrices $A$, a row matrix and $B$, a column matrix.

In general, an element $c_{ij}$ of matrix $C$ is a result of the multiplication of the vector in the $i$th row vector of $A$ with the vector in the $j$th column of $B$.

$$c_{ij} = \bold{a_i} . \bold{b_j}$$ ...