Understanding the Design of a Convolutional Neural Community

Final Up to date on July 13, 2022

Convolutional neural networks have been discovered profitable in laptop imaginative and prescient functions. Varied community architectures are proposed and they’re neither magical nor arduous to know.

On this tutorial, we’ll make sense of the operation of convolutional layers and their position in a bigger convolutional neural community.

After ending this tutorial, you’ll study:

  • How convolutional layers extract options from picture
  • How completely different convolutional layers can stack as much as construct a neural community

Let’s get began.

Understanding the Design of a Convolutional Neural Community
Picture by Kin Shing Lai. Some rights reserved.


This text is break up into three sections; they’re:

  • An Instance Community
  • Exhibiting the Function Maps
  • Impact of the Convolutional Layers

An Instance Community

The next is a program to do picture classification on the CIFAR-10 dataset:

This community ought to be capable to obtain round 70% accuracy in classification. The pictures are in 32×32 pixels in RGB colour. They’re in 10 completely different courses, which the labels are integers from 0 to 9.

We are able to print the community utilizing Keras’ abstract() operate:

On this community, the next shall be proven on the display screen:

It’s typical in a community for picture classification to comprise of convolutional layers at early stage, with dropout and pooling layers interleaved. At later stage, the output from convolutional layers are flattened and processed by some totally linked layers.

Exhibiting the Function Maps

Within the above community, we used two convolutional layers (Conv2D). The primary layer is outlined as follows:

which implies the convolutional layer can have a 3×3 kernel and apply on an enter picture of 32×32 pixels and three channels (the RGB colours). The output of this layer shall be 32 channels.

To make sense of the convolutional layer, we are able to take a look at its kernel. The variable mannequin holds the community and we are able to discover the kernel of the primary convolutional layer with the next:

and this prints:

We are able to inform that mannequin.layers[0] is the proper layer by evaluating the title conv2d from the above output to the output of mannequin.abstract(). This layer has a kernel of form (3, 3, 3, 32), that are respectively the peak, width, enter channels, and output function maps.

Assume the kernel is a NumPy array ok. A convolutional layer will take its kernel ok[:, :, 0, n] (a 3×3 array) and apply on the primary channel of the picture. Then apply ok[:, :, 1, n] on the second channel of the picture, and so forth. Afterwards, the results of the convolution on all of the channels are added as much as grow to be function map n of output, which n on this case will run from 0 to 31 for the 32 output function maps.

In Keras, we are able to extract the output of every layer utilizing an extractor mannequin. Within the following, we create a batch with one enter picture and ship to the community. Then we take a look at the function maps of the primary convolutional layer:

The above code will print the function maps like the next:

That is comparable to the next enter picture:

We are able to see that we name them the function maps as a result of they’re highlighting sure options from the enter picture. A function is recognized utilizing a small window (on this case, over a 3×3 pixels filter). The enter picture has 3 colour channels. Every channel has a unique filter utilized, which their outcomes are mixed for an output function.

We are able to equally show the function map from the output of the second convolutional layer, as follows:

Which exhibits the next:

From the above, you possibly can see that the options extracted are extra summary and fewer recognizable.

Impact of the Convolutional Layers

Crucial hyperparameter to a convolutional layer is the dimensions of the filter. Often it’s in a sq. form and we are able to take into account that as a window or receptive area to have a look at the enter picture. Due to this fact, the upper decision of the picture, we’d anticipate a bigger filter.

However, a filter too giant will blur the detailed options as a result of all pixels from the receptive area via the filter shall be mixed into one pixel on the output function map. Due to this fact, there’s a commerce off for the suitable dimension of the filter.

Stacking two convolutional layers (with out another layers in between) is equal to a single convolutional layer with bigger filter. However it is a typical design these days to make use of two layers with small filters stacked collectively somewhat than one bigger with bigger filter, as there are fewer parameters to coach.

The exception can be convolutional layer with 1×1 filter. It’s normally discovered as the start layer of a community. The aim of such a convolutional layer is to mix the enter channels into one somewhat than reworking the pixels. Conceptually, this may convert a colour picture into grayscale, however normally we make a number of methods of conversion to create extra enter channels than merely RGB for the community.

Additionally word that within the above community, we’re utilizing Conv2D, for a 2D filter. There may be additionally a Conv3D layer for a 3D filter. The distinction is whether or not we apply the filter individually for every channel or function map, or to contemplate the enter function maps stacked up as a 3D array and apply a single filter rework it altogether. Often the previous is used as it’s extra affordable to contemplate no explicit order the function maps must be stacked.

Additional Studying

This part gives extra assets on the subject if you’re trying to go deeper.




On this publish, you’ve gotten seen how we are able to visualize the function maps from a convolutional neural community and the way it works to extract the function maps

Particularly, you realized:

  • The construction of a typical convolutional neural networks
  • What’s the impact of the filter dimension to a convolutional layer
  • What’s the impact of stacking convolutional layers in a community

Develop Deep Studying Tasks with Python!

Deep Learning with Python

 What If You Might Develop A Community in Minutes

…with just some strains of Python

Uncover how in my new E book:

Deep Studying With Python

It covers end-to-end tasks on matters like:

Multilayer PerceptronsConvolutional Nets and Recurrent Neural Nets, and extra…

Lastly Carry Deep Studying To

Your Personal Tasks

Skip the Lecturers. Simply Outcomes.

See What’s Inside

Leave a Comment