...
/Role of Pooling Layers in Convolutional Network Regularization
Role of Pooling Layers in Convolutional Network Regularization
Understand how pooling layers introduces invariance, balancing equivariance and network efficiency.
We'll cover the following...
Pooling brings “invariance” to a convolutional network. If a function is invariant, its output is unaffected by any translational change in the input. A pooling operation draws a summary statistic from the convolution output. This replaces the overrepresentative convolution output with a single or a few summary statistics. Pooling, therefore, further regularizes the network and maintains its statistical efficiency.
However, like equivariance, an invariance to translation is sometimes counterproductive. Invariance makes the network blind to the location of patterns in the input. This sometimes leads to a network confusing the original input with its distortions.
Pooling provides a lever to modulate the network between equivariance and invariance. Somewhere between the two is usually optimal.
As a result, a pooling layer complements a convolutional layer. Moreover, pooling does not have trainable parameters. Therefore, it doesn’t add computational overhead.
Note: A pooling layer provides a mechanism to regularize a convolutional network without adding computational overhead.
Regularization via invariance
Invariance is the opposite of equivariance. If a function is invariant, its output is unaffected by any translational change in the input.
Pooling brings invariance to a convolutional network. A pooling operation summarizes the convolutional output into a statistic(s), called a summary statistic. For example, the maximum summary statistic is returned in MaxPool. In doing so, the granular information about an object’s location is lost.
This causes the network to become invariant to the location of an object in the input. The phenomenon is explained in the illustrations below.
The illustrations above show that the Maxpool operand takes in the output from convolution and emits the maximum value. Consequently, despite the letter “2” being at the top-left or bottom-right, the output from pooling is the same .
In effect, pooling regularizes the network by reducing the spatial size of the ...