Top 10 Deep Learning Algorithms to know – Part 1

 

Most machine learning algorithms perform effectively on datasets of a few hundred features or columns. However, an unstructured dataset, such as one from an image, contains such a vast number of features that working with ML algorithms becomes laborious or altogether impractical.

Deep learning algorithms compensate where ML algorithms fail. DL algorithms are equipped to efficiently handle unstructured datasets such as images. As the image passes through each neural network layer, deep learning algorithms learn more about it. The primitive layers learn to recognise low-level elements like edges, while advanced layers combine those features into a more holistic picture.

In this Part 1 of the two series blog, we are covering the working of 5 algorithms viz.

1.    Convolutional Network

2.    Recurrent Neural Network

3.    Long Short-Term Memory Network

4.    Generative Adversarial Network

5.    Radial Basis Function Network

 

1.   Convolutional Neural Network

What is it?

CNNs, also known as ConvNets, are networks of neurons with learnable weights and biases. Each neuron takes some inputs, predominantly an image, does a dot product, and then executes a non-linearity if desired. The assumption here is that inputs are images, allowing us to embed specific attributes into the architecture. As a result, the forward function is more efficient to construct, and the number of parameters in the network is greatly reduced.

How does it Work?

     

Figure 1 - Convolutional Neural Network; Source

 

Convolutional Neural Networks have the following layers:

●     Convolutional - This is the first step in the process of extracting valuable characteristics from an image. The convolution action is performed by many filters in a convolution layer. Every image is viewed as a pixel value matrix.

●     ReLU Layer - The rectified linear unit is abbreviated as ReLU. After the feature maps have been obtained, they must be transferred to a ReLU layer. ReLU performs an element-by-element procedure, setting all negative pixels to zero. It allows the network to become non-linear, and the result is a rectified feature map.

●     Pooling - Pooling layers (or downsampling), is a dimensionality reduction technique that reduces the number of parameters in the input. The pooling process masks a filter across the entire input; however, this filter does not have any weights. Instead, the kernel uses an aggregation function to populate the output array from the values in the receptive field. Types of pooling – Max pooling & Average Pooling

●     Fully Connected Layer - The full-connected layer's name is self-explanatory. In partially linked layers, the pixel values of the input image are not connected to the output layer. Each node in the output layer is connected directly to a node in the previous layer. This layer performs classification tasks based on the features retrieved by the previous layers and their various filters. While convolutional and pooling layers typically utilise ReLu functions to categorise inputs, FC layers generally use a Softmax activation function to categorise inputs correctly, giving a probability from 0 to 1.

2.   Recurrent Neural Networks (RNNs)

What is it?

A recurrent neural network (RNN) is one of the artificial neural network algorithms that work with time series or sequential data. These deep learning algorithms are often employed for ordinal or temporal issues like language translation, natural language processing (NLP), speech recognition, and image captioning, and they're used in popular apps like Siri, voice search, and Google Translate.

How does it Work?

 

Figure 2 - Recurrent Neural Network; Source

 

You'll need a working knowledge of "conventional" feed-forward neural networks and sequential data (or time-series data – e.g. – financial data or DNA data) to fully comprehend RNNs. RNNs and feed-forward neural networks channel derived their name is from how they process information

The information in a feed-forward neural network (FFNN) only flows in a unidirectional method: from the input layer to the output layer, passing through the hidden layers. The data travels through the network, never passing through the same node twice.

FFNN have no recollection of the information they receive and are poor predictors of what will happen next. An FFN has no concept of time order because it only analyses the current input. Except for its training, it has no memory of past events.

The information in RNN cycles via a loop. When it makes a judgement, it considers the current input as well as what it has learnt from prior inputs.

Types of RNNs

●     One to One

●     One to Many

●     Many to One

●     Many to Many

3.   Long Short-Term Memory Network

What is it?

Long short-term memory networks (LSTMs) are a type of recurrent neural network that enhances the memory capacity of the network. This makes it suitable for learning from significant data separated by lengthy periods.

How does it Work?

Figure 4 - LSTM Network; Source

 

The control flow of an LSTM is comparable to that of a recurrent neural network. It processes data and passes the information on as it moves along. What differs is the operations within the LSTM's cells. The LSTM uses these procedures to remember or forget information.

RNNs can recall inputs for a long time with the help of LSTMs. This is because of LSTMs that store information in memory like that of a computer. The LSTM can read, write, and delete data from its memory.

This memory can be thought of as a gated cell, with gated indicating that the cell selects whether to store or erase information (i.e., whether to open the gates) based on the value it gives to the data. The importance of the information is determined by the weights it carries. This simply means that the algorithm learns to decide whether the information is relevant or not.

There are three gates in an LSTM: input, forget, and output. These gates determine whether incoming input should be allowed (input gate), whether it should be deleted because it isn't important (forget gate), or whether it should have an impact on the output at the current timestep (output gate).

4.   Generative Adversarial Networks (GANs)

What is it?

GANs (generative adversarial networks) are computational structures that pit two neural networks against one other (thus the term "adversarial") to derive new, synthetic examples of data that can pass for real data. They're commonly utilised in the picture, video, and voice generation.

Because GANs can learn to replicate any data distribution, they have enormous potential for both good and evil. In other words, GANs can be taught to generate universes that are uncannily identical to our own in any domain: images, music, speech, and literature. They are, in a sense, robot artists, and their work is stunning – even heart-breaking. They can, however, be used to create fake media content, and this is the technology that underpins Deepfakes.

How does it Work?

To comprehend GANs, you must first understand how generative algorithms function, which can be done by contrasting them with discriminative algorithms. Discriminative algorithms attempt to classify input data; that is, they anticipate a label or category to which an instance of data belongs based on the features of that data.

Discriminative algorithms map features to labels (read attributes to function). Generative algorithms can be thought of as doing the exact opposite. Rather than predicting a label based on traits, they try to predict features based on a label.

 

Figure 5 – GAN; Source

The following are the steps taken by a GAN:

●     The generator generates an image from random numbers.

●     This image, together with a stream of photos from the real dataset, is sent into the discriminator.

●     The discriminator accepts both actual and false photos and returns probabilities, which range from 0 to 1, with 1 indicating authenticity and 0 indicating fake.

As a result, you have a two-way feedback loop:

●     The discriminator network is in a feedback loop with the real images.

●     The generator network is in a feedback loop with the discriminator network.

5.   Radial Basis Function Networks (RBFNs)

What is it?

An RBFN is an artificial neural network containing an input layer, a hidden layer, and an output layer. Hidden neurons make up the RBF's Hidden layer, and their activation function is a Gaussian function. The hidden layer provides a signal that corresponds to an input vector in the input layer, and the network develops a response in response to this signal.

How does it Work?

Three primary components of a Radial Basis Function Network are:

●     The input vector to be classified,

●      The Radial Basis Function neurons, and

●     The output nodes.

RBFN comprises Radial Basis Function neurons grouped in a layer. A "prototype" vector is stored by each neuron in the layer to store information from the training data. When the RBFN processes an input vector, each neuron in the RBFN layer compares its prototype vector to the input and the output is a number between 0 and 1, indicating similarity. A value of 1 indicates that the input vector matches the prototype, and the values are closer to 0 as the dissimilarity between input and prototype increases.

Figure 6 – RBFN; Source

 

Now that each neuron has provided a value of similarity, the output nodes take a weighted total of the values. The output nodes can polarise the data by weighting the sums, making the classification more distinct. To represent their separate categories, two output nodes will have two different weights. A positive weight can be assigned to one node and a negative weight to the other. An input vector can be processed by an RBF layer, which assigns a similarity score, and then separately categorised by a set of output nodes using this network architecture.