DeapSECURE module 4: Deep Learning (Neural Network)

Key Points

Introduction to Machine Learning & Deep Learning	Deep learning is a subset of machine learning, which is a subset of AI. Deep learning is an advanced form of machine learning that uses neural networks, which mimic the neurons in the human brain, to collect data, learn and modify the model as it learns more.
Deep Learning to Identify Smartphone Applications	Malicious apps (malware) have become a prevalent tool for compromising mobile devices, stealing personal information, or spying on a user’s activities. Researchers have leveraged artificial intelligence (AI) / machine learning (ML) to keep up with increasing security challenges. Large amounts of data are critical to train and validate accurate and effective ML-based cybersecurity techniques. ML models can be used to distinguish smartphone apps and potentially identify malware.
Overview of Deep Neural Network Concepts	Deep neural networks have linear and nonlinear parts. Activation functions add the nonlinear component to the model. Forward propagation calculates the output based on the current weights. Backpropagation is used to adjust the network parameters. An HPC can be adopted to speed up the training and inference processes.
An Introduction to Keras with Binary Classification Task	Keras is an easy-to-use high-level API for building neural networks. Main parts of a neural network layer: number of hidden neurons, activation function. Main parts of a neural network model: layers, optimizer, learning rate, loss function, performance metrics.
Classifying Smartphone Apps with Keras	On KERAS, we can easily build the network by defining the layers and connecting them together.
Tuning Neural Network Models for Better Accuracy	Neural network models are tuned by tweaking the architecture and tuning the training hyperparameters.
Effective Deep Learning Workflow on HPC	How scripting works by converting the notebook to job scripts Build a simple toolset/skillset to create, launch, and manage the multiple batch jobs. Use this toolset to obtain the big picture result after analyzing the entire calculation results as a set. Use Jupyter notebook as the workflow driver instead of using it to do the heavy-lifting computations on it.
Post-Analysis for Modeling Tuning Experiments	Post-analysis focuses on analyzing the results (of a model) to better understand the behavior and improve its performance.
Using Multicore CPUs or GPUs for KERAS Computation	First key point. Brief Answer to questions. (FIXME)
Dealing with Issues in Data and Model Training	Data issues like missing values, imbalance, and errors lead to biased models and poor predictions. Metrics like precision, recall, and F1 score provide class-specific insights into model performance. Overfitting and underfitting can be detected by comparing training and validation metrics.
Deep Learning in the Real World	Layer design

Glossary

Activation Function: A mathematical function that introduces non-linearity into a (NN) model.
Backpropagation: An algorithm that updates/corrects the weights so as to bring the predicted outcome closer to the expected outcome.
Features: Are attributes of data used for training and testing ML algorithms.
Forward Propagation: Calculates the output based on the current weights (and biases).
Hidden Layer: A layer of (artificial) neurons between the input and output layers. The layers introduce nonlinearity to the model.
Hyperparameters: “Settings” or variables that are set beforehand that controls the learning process (of a model). Hyperparameters govern how the model learns and adjusts the parameters. Hyperparameters for a neural network can include the number of layers and the number of neurons in each layer, the learning rate, the activation function, the optimizer, and many more.
Inference: Uses a trained model to make predictions or “inferences” on new, unseen data. In other words, deploying the model to make predictions on new, unseen data.
Input Layer: The first layer of a neural network that receives the input data and passes it to the next layer.
Loss Function: Measures how well a model’s (current) output/predictions compare with the actual target values (also referred to as ground-truth labels). Also known as a cost or error function.
Parameters: Internal variables that are adjusted during training. Refers to weights and bias.
Optimizer: An algorithm used to iteratively improve the model during training. It adjusts the parameters (weights and bias) according to the loss.
Output Layer: The final layer in a neural network that generates the prediction or output.
Shape: The shape of a tensor or vector refers to the dimensionality. A 1-D shaped tensor (a vector) represents the sequence length. Can also be referred to as “size.”

DeapSECURE module 4: Deep Learning (Neural Network)

Key Points

Glossary

Further Reading

Other Courses

Building neural networks

Optimizers