Introduction to Machine Learning & Deep Learning

Overview

Teaching: 0 min
Exercises: 0 min

Questions

• What is deep learning?

• What are the top deep learning algorithms?

• What are the differences between Artificial Intelligence (AI) vs Machine Learning (ML) and Deep Learning (DL)?

Objectives

• Learning general concepts of machine learning, deep learning and their relationship with artificial intelligence.

• Diving deeper into deep learning concepts and algorithms and the tools necessary.

Introduction: Artificial Intelligence, Machine Learning, and Deep Learning

Artificial intelligence (AI) has recently gained a lot of popularity; many technologies containing AI have been deployed, which have tremendously improved our lives.

In this episode, we will attempt clarify the difference between artifical intelligence (AI), machine learning (ML), and deep learning (DL). These three items are closely related, but they are not fully identical or interchangeable.

Artificial Intelligence

For many years, scientists dreamed of enabling computers to think for themselves. According to Britannica, AI is:

“the ability of a digital computer or computer-controlled robot to perform tasks commonly associated with intelligent beings. The term is frequently applied to the project of developing systems endowed with the intellectual processes characteristic of humans, such as the ability to reason, discover meaning, generalize, or learn from past experience.”

The Oxford English Dictionary gives the following definition for AI:

“The capacity of computers or other machines to exhibit or simulate intelligent behaviour; the field of study concerned with this. In later use also: software used to perform tasks or produce output previously thought to require human intelligence”

Machine Learning

Machine learning (ML) is a subset of AI that provides computer systems the ability to automatically learn and improve from experience without being explicitly programmed.

Emerj collected several definitions of machine learning by well-known parties and came up to the following conclusion:

Machine Learning is the science of getting computers to learn and act like humans do, and improve their learning over time in autonomous fashion, by feeding them data and information in the form of observations and real-world interactions.

Machine learning is about discovering patterns from data (in numerical form) and modeling these patterns. In other words, mapping features (inputs) to outputs. These mappings utilize various mathematical (statistical, probabilistic, non-linear, etc.) functions and methods. Supervised and unsupervised learning are the main two approaches to machine learning. Supervised ML trains on labeled input data to predict outcomes. Unsupervised ML trains on unlabeled data to predict outcomes.

Module 3, Lesson 1: Introduction to Machine Learning goes into more depth on machine learning.

Deep Learning

Deep learning is a subset/specific branch of machine learning. IBM defines deep learning as:

“a subset of machine learning that uses multilayered neural networks, called deep neural networks, to simulate the complex decision-making power of the human brain.”

Deep learning (DL) aims to achieve a high level of performance (approaching, or even exceeding, human intelligence capabilities) based on a specific model architecture known as artificial neural networks (often simply known as neural networks (NN)). Similar to machine learning, deep learning algorithms first require training to be able to learn patterns and tasks that are being assigned.

As we shall see later in this lesson, NN models are strongly inspired by the neural structure of the human brain and how it works. DL is much more sophisticated and powerful than traditional ML algorithms. It is able to perform intelligent tasks beyond what is possible with traditional ML, i.e. tasks such as accurate recognition of objects in images and videos, handwriting and speech recognition, and interacting with humans (e.g. chatbots). However, DL is very computationally expensive.

Relationships between Artificial Intelligence, Machine Learning and Deep Learning

Figure: Relationships between Artificial Intelligence, Machine Learning, and Deep Learning.

Contrast and Comparison between Machine Learning and Deep Learning

(Source)

Overview of Machine Learning

The machine learning process can be divided into five sections:

1) The input is data generated and collected in real life, such as voices, images, etc.;

2) Features are attributes of data used for training and testing ML algorithms and are usually obtained by feature abstraction, usually requiring human interaction (ideally by someone with domain knowledge);

3) The output is any value(s) one wants to obtain from the input;

4) Learning algorithms or models distinguish the patterns of a feature by adjusting a set of parameters through training. Briefly, machine learning models, which are essentially mathematical functions, map between the features (inputs) and the outputs.

5) The inference, is the process of using a trained model to predict or classify on unseen data. It is conducted based on the determined parameters via learning algorithms.

Figure: Overview of machine learning.

Applications of Machine Learning

Machine learning has various applications, including:

• Image recognition (e.g. facial recognition for Facebook or Google Photos or the face unlock feature on smartphones)

• Movie or product recommendation (e.g. app store or play store application recommendations)

• Fraud detection

• Medical diagnosis

• Autonomous driving

Future applications include things like self-driving cars, health monitoring, and many more.

In cybersecurity areas, machine learning has been used or researched for these purposes:

• Detection of malware, both existing and novel types

• Detection of malicious URLs

• Detection of network attacks, intrusions or suspicious activities

Discussion Topic

Discuss with your partner an example for each application listed above, and consider what the input and output could be for each.

Overview of Deep Learning

Deep learning enables a machine to constantly adapt to its surroundings and make changes as needed. This ensures versatility of operation, making it possible for a machine to efficiently analyze problems through its hidden layer architecture that would otherwise be far too complex to address through manual programming. Deep learning offers an advantage when handling huge volumes of unstructured data, as it does not require any labels and does not require a human to do feature engineering.

Deep neural networks consist of multiple layers of interconnected nodes, each building upon the previous layer to refine and optimize the resulting prediction or categorization. Over time, the algorithm becomes gradually more accurate. The input and output layers of a deep neural network are called visible layers. The input layer is where the deep learning model ingests the data for processing, and the output layer is where the final prediction or classification is made. Layers between the input and output layers are called hidden layers. Feature extraction and learning are automated within these layers.

(Source)

The figure below is an example of a deep learning network.

Figure: Overview of deep learning.

There are many variants of neural network algorithms, each serving specific goals. The following list contains some of the more popular ones:

• Fully-connected neural network, also known as Multilayer Perceptrons (MLPs)
• Convolutional Neural Networks (CNNs)
• Recurrent Neural Networks (RNNs)

We will return to these names, what they are in a nutshell, and their applications in the last lesson of this module.

Note, though NN models are usually thought of as DL models, DL models utilize three or more hidden layers. NN models with less than three are not “deep” (and are called “shallow”) and therefore are categorized as traditional machine learning models.

Deep learning tools and services

There are many deep learning tools and services currently available. Below is the top 10 Deep Learning frameworks in 2025:

• TensorFlow
• Keras
• Caffe
• Pytorch
• Microsoft Cognitive Toolkit (previously CNTK)
• MXNet
• Theano
• Deeplearning4j

For further reading, consider the following:

• Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey
• Comparison of deep learning software (Wikipedia)

Introduction to TensorFlow and Keras

In this lesson, we will provide a hands-on introduction to NN using two closely related Python libraries: TensorFlow and Keras.

TensorFlow is a powerful computation framework that provides a convenient way to express complex neural-network low-level mathematics involving series of tensors (which, briefly, are high-dimensional “matrices”). In this training module, we will leverage the capabilities of TensorFlow via a higher-level API named Keras.

Keras is a high-level software framework used to perform training and inference for neural networks. Originally, Keras was a stand-alone project (https://keras.io) that provided a uniform, high-level, user-friendly API for various lower-level neural network libraries: TensorFlow, Microsoft Cognitive Toolkit (CNTK), and Theano. (Theano is now unmaintained, so it is not recommended for new projects.) Keras requires the use of a low-level neural network library (TensorFlow, Theano or CNTK), which provides the actual computational capabilities. We will use only the TensorFlow backend in this workshop.

With Keras, construction of a neural network has become tremendously easier. As we will soon see, Keras users need not be overburdened by the different conventions and bookkeeping rules that are encountered when using lower-level libraries. They can instead focus on the structure of the network, providing almost 1:1 mapping to human-friendly neural network diagrams.

The popularity and adoption rate of Keras was so high that TensorFlow developers announced on January 2017 that they adopted KERAS as the high-level API for TensorFlow. As a result, there has been an integrated KERAS API since TensorFlow version 1.5.

Figure: Keras-Tensorflow Chart.

Next section

Next, we will introduce the structure of deep learning networks. We will focus on a popular and powerful branch of deep learning called Artificial Neural Network (ANN) or, more often, as Deep Neural Network (DNN), or simply Neural Network (NN).

(TODO) Explain the roadmap of the episodes, incl the “problem” we want to answer (ie. smartphone app classification), and activities we will be doing.

Key Points

• Deep learning is a subset of machine learning, which is a subset of AI.

• Deep learning is an advanced form of machine learning that uses neural networks, which mimic the neurons in the human brain, to collect data, learn and modify the model as it learns more.