Neural Network Tutorials - Herong's Tutorial Examples - v1.22, by Herong Yang
What Is Convolutional Layer
This section provides a quick introduction of convolutional layer, which convolves a feature pattern with the full set of input features to promote the given pattern.
What Is Convolutional Layer? Convolutional Layer is a layer of neural nodes in neural network that convolves a feature pattern with the full set of input features. The main purpose of a convolutional Layer is to promote a given pattern in the sample by enhancing feature structures similar to the pattern and hiding other feature patterns.
A convolutional layer is considered as a filter since it filters input features to enhance the given feature pattern.
The feature pattern is called the filter kernel, which is usually made of a small number of neighboring features. Then the kernel is convolved with features in local neighbor areas sequentially throughout the entire input feature set to generate outputs.
Here is a good diagram that demonstrates how a 2 dimensional filter kernel is convolved with a 2 dimensional input feature set (source: redcatlabs.com). In this case, each input feature is the color intensity of a pixel in an image.
The filter kernel used in the above picture is called Emboss kernel, which represent a feature pattern of a diagonal edge in images. The Emboss kernel is made of 9 neighboring pixels as show below:
| 4 0 0 |
k = | 0 0 0 |
| 0 0 -4 |
When the Emboss kernel is convolved with the input image, the output can be viewed as a transformed image. The color intensity of each pixel in the transformed image is the weighted sum of the pixel's local neighbor area of the original image with kernel image:
yi,j = k1,1*xi-1,j-1 + k1,2*xi-1,j + k1,3*xi-1,j+1
+ k2,1*xi, j-1 + k2,2*xi ,j + k2,3*xi ,j+1
+ k3,1*xi+1,j-1 + k3,2*xi+1,j + k3,3*xi+1,j+1
When convolving a kernel on a pixel on the image edges, special treatments are needed, such as padding the original image with extra pixels, or skip convolving on image edges.
There are a number of well-known image filter kernel that can be used to capture different image feature patterns. For example, the vertical Sobel kernel, can be used to capture vertical edges as shown in the following example (source: victorzhou.com).
Vertical Sobel Kernel:
| -1 0 1 |
k = | -2 0 2 |
| -1 0 1 |
The horizontal Sobel kernel, can be used to capture horizontal edges as shown in the following example (source: victorzhou.com).
Horizontal Sobel Kernel:
| 1 2 1 |
k = | 0 0 0 |
| -1 -2 -1 |
Multiple convolutional layers can be applied to the same input layer to capture different feature patterns.
For more details on convolutional layer, read "CNNs, Part 1: An Introduction to Convolutional Neural Networks" by Victor Zhou at https://victorzhou.com/blog/intro-to-cnns-part-1/. You can follow his code example to build your own CNN model with NumPy library for image processing.
Table of Contents
Deep Playground for Classical Neural Networks
Building Neural Networks with Python
Simple Example of Neural Networks
TensorFlow - Machine Learning Platform
PyTorch - Machine Learning Platform
►CNN (Convolutional Neural Network)
What Is CNN (Convolutional Neural Network)
RNN (Recurrent Neural Network)
GAN (Generative Adversarial Network)