Introduction Deep Learning

Deep learning is a branch of machine learning which is completely based on artificial neural networks, as the neural network is going to mimic the human brain so deep learning is also a kind of mimic of the human brain. In deep learning, we don’t need to explicitly program everything. The concept of deep learning is not new. It has been around for a couple of years now. It’s on hype nowadays because earlier we did not have that much processing power and a lot of data. As in the last 20 years, the processing power increases exponentially, deep learning and machine learning came in the picture.

A formal definition of deep learning is- neurons

"Deep learning is a particular kind of machine learning that achieves great  power and flexibility by learning to represent the world as a nested hierarchy of concepts, with each concept defined in relation to simpler concepts, and more abstract representations computed in terms of less abstract ones".

In the human brain approximately 100 billion neurons all together this is a picture of an individual neuron and each neuron is connected through thousand of their neighbours.

The question here is how do we recreate these neurons in a computer. So, we create an artificial structure called an artificial neural net where we have nodes or neurons. We have some neurons for input value and some for output value and in between, there may be lots of neurons interconnected in the hidden layer.

Architectures :

1. Deep Neural Network – It is a neural network with a certain level of complexity (having multiple hidden layers in between input and output layers). They are capable of modelling and processing non-linear relationships.
2. Deep Belief Network(DBN) – It is a class of Deep Neural Network. It is multi-layer belief networks.
   Steps for performing DBN :
   a. Learn a layer of features from visible units using the Contrastive Divergence algorithm.
   b. Treat activations of previously trained features as visible units and then learn features of features.
   c. Finally, the whole DBN is trained when the learning for the final hidden layer is achieved.
3. Recurrent (perform the same task for every element of a sequence) Neural Network – Allows for parallel and sequential computation. Similar to the human brain (large feedback network of connected neurons). They are able to remember important things about the input they received and hence enables them to be more precise.