DeapSECURE module 3: Machine Learning

Machine learning methods are divided into two classes: supervised and unspervised.

Supervised ML methods train models based on given data with labels/outcomes/objectives.

Unsupervised ML methods aims at finding structure/pattern without labeling the data.

Scikit-Learn provides machine learning capabilities for Python.

Pandas provides data handling and analytic tools for Python.

Understanding a machine learning problem.

Data preparation steps can be grouped into two categories: data cleaning and feature selection.

Data cleaning steps include removal of irrelevant or duplicate data as well as handling of missing data.

Feature selection aims identify features with the most predictive power for machine learning?

Relevant features are selected through careful observation and performing correlation and simple grouping analysis.

The output for classification can be encoded into binary bits.

Logistic regression and decision tree are two examples of classic machine learning algorithms.

Learn to fit the model with training set and test the model with model evaluation function.

The key methods for machine learning model tuning include: feature selection and model hyperparameter adjustments.

Spark MLlib provides a way to apply machine learning algorithms on extremely large datasets.

Machine learning is not a blackbox and therefore should be used with proper care.

Key steps preceding machine learning: Data loading, exploration, cleaning, and input preparation