Intro to Deep Learning

What is Deep Learning?

“Deep learning is a particular kind of machine learning that achieves great power and flexibility by learning to represent the world as nested hierarchy of concepts, with each concept defined in relation to simpler concepts, and more abstract representations computed in terms of less abstract ones.”

Deep Learning is a subfield of machine learning which its algorithms inspired by the structure and function of the human brain. This Algorithm takes raw data as an input and process the data through some layers of the nonlinear transformation of the input data to compute the output. The layers are made of nodes. A node is just a place where computation happens, loosely patterned on a neuron in the human brain, which fires when it encounters sufficient stimuli. A node combines input from the data with a set of coefficients, or weights, that either amplify or dampen that input, thereby assigning significance to inputs for the task the algorithm is trying to learn.

Here's a few key trends in deep learning history:

• Deep learning has had a long and rich history, but has gone by many names,reflecting different philosophical viewpoints, and has waxed and waned inpopularity.
• Deep learning has become more useful as the amount of available trainingdata has increased.
• Deep learning models have grown in size over time as computer infrastructure(both hardware and software) for deep learning has improved.
• Deep learning has solved increasingly complicated applications with increasingaccuracy over time.

These are examples of deep application: voice recognition, machine translation, , automatic text generation.etc.

Where can we apply deep learning?

There are so monay domain where we can apply our deep learning application such as:

• Computer Vision: for applications like vehicle number plate identification and facial recognition.
• Information Retrieval: for applications like search engines, both text search, and image search.
• Marketing: for applications like automated email marketing, target identification
• Medical Diagnosis: for applications like cancer identification, anomaly detection
• Natural Language Processing: for applications like sentiment analysis, photo tagging
• Online Advertising, etc

Deep Learning Techniques

These are the popular deep learning techniques:

Multilayer Perceptron (MLP)

A multilayer perceptron (MLP) is a class of feedforward artificial neural network and it consists of three or more layers (an input and an output layer with one or more hidden layers) of nonlinearly-activating nodes.

Convolutional Neural Networks (CNNs)

CNNs are neural networks that employ the convolution operation (instead of a fully connected layer) as one of its layers. CNNs are often used to recognize objects and scenes, and perform object detection and segmentation. They learn directly from image data, eliminating the need for manual feature extraction.

Recurrent Neural Networks (RNNs)

RNNs, are a type of artificial neural network that add additional weights to the network to create cycles in the network which is basically using information from a previous forward pass over the neural network. RNNs are often applied to problems wherein the input data on which the predictions are to be made is in the form of a sequence .

Long Short-Term Memory (LSTMs)

LSTMs are a special kind of RNN, capable of learning long-term dependencies in sequence prediction problems. It often used in complex problem domains like machine translation, speech recognition, and more.

Deep Learning Vs Other Machine Learning Algorithms

Deep learning also has same basic definition as machine learning which:

input dataX into trained function F(x) and return result y. However, there are few points that we will compare these two techniques:

Data dependencies:

the most important difference between deep learning and other traditional machine learning is the amount of training data. When data is small, deep learning not learn well because it need large amount of data in order understant it perfectly. However, traditional machine learning needs little amount data to train on because its algorithm can perform well or not is based on a human expert who tuning the algorithm parameters.

Hardware dependencies:

Because deep learning need to train on large amount of data which mean lot of calculation, it needs to use high performance hardware for the operation. On the other hand, other machine learning algorithm can work on low performance hardware since it use little data.

Execution time:

More data mean more time as well.

Feature engineering:

In Machine learning, most of the applied features need to be identified by an expert and then hand-coded as per the domain and data type.

Deep learning algorithms try to learn high-level features from data by it self.

Libraries

There are so many deep learning libraries to choose which are the good professional libraries that you can use to build your toy and production project. And here they are:

• Tensorflow
• Keras
• Theano
• Torch
• Caffe
• DeepLearning4J
• MXNet
• BigDL
• ...

Resources

• https://www.coursera.org/specializations/deep-learning
• http://www.deeplearningbook.org/contents/intro.html
• http://neuralnetworksanddeeplearning.com/chap1.html
• https://machinelearningmastery.com/what-is-deep-learning/
• https://www.xenonstack.com/blog/log-analytics-with-deep-learning-and-machine-learning
• https://datahub.packtpub.com/deep-learning/top-10-deep-learning-frameworks/
• https://machinelearningmastery.com/popular-deep-learning-libraries/
• https://www.analyticsvidhya.com/blog/2017/04/comparison-between-deep-learning-machine-learning/
• Deep Learning with Python - A Hands-on Introduction - 1E (2017)

Summary

In this post, we talk about what is deep learning then we list some deep learning techniques such as:

• Multilayer Perceptron (MLP)
• Convolutional Neural Networks (CNNs)
• Recurrent Neural Networks (RNNs)
• Long Short-Term Memory (LSTMs)

We also discuss about the differences between deep learning and other machine learning algorithm. Finally, we list the libraries where we can use to build our own deep learning APP.

In next article, We'll use one of those libraries to build our deep learning project.

Stay turned.