Kernels in Convolutional Network Operations

Convolution

Convolution is a filtration process, a filter is made of the kernel(s), and a kernel is a mathematical representation of a filter.

Kernels can be either continuous or discrete functions. Discrete kernels are commonly used in convolutional networks due to their flexibility. This choice is similar to fitting an empirical distribution in statistical analysis. Discrete kernels make convolutional networks robust to the distribution of input features.

Suppose, $f$ is a discrete kernel and $g$ is a mono-channel image, then the convolution of $f$ and $g$ is expressed as:

$$f∗g= \bigg[\sum_{u=−∞}^{+∞}\sum_{v=−∞}^{+∞}f(u,v)g(x−u,y−v)\bigg] ,∀x,y.$$

Here, $f$ is a two-dimensional kernel because $g$ is two-dimensional. The dimensions here are spatial. The term dimension can be confused with the number of features for statisticians. Therefore, the axis denotes a spatial dimension.

A convolution operation involves sweeping the input. The sweeping occurs along each axis of the input. Therefore, because the input $g$ has two axes, the convolution operation in the equation above is a double summation along both.

A mono-channel image is a two-axes $m × n$ matrix. For illustration, suppose the image is a $5 × 5$ matrix:

$$G=\begin{bmatrix} g_{11} & g_{12} & g_{13} & g_{14} & g_{15} \\ g_{21} & g_{22} & g_{23} & g_{24} & g_{25} \\ g_{31} & g_{32} & g_{33} & g_{34} & g_{35} \\ g_{41} & g_{42} & g_{43} & g_{44} & g_{45} \\ g_{51} & g_{52} & g_{53} & g_{54} & g_{55} \end{bmatrix}$$ ...