With the explosion in use and popularity of ChatGPT and the proliferation of AI tools I thought I’d attempt to demystify Machine Learning (ML) and give some simple insights into what it actually is and how it works.

In simple terms, machine learning is a subset of artificial intelligence (AI) that allows machines to automatically learn and improve from experience without being explicitly programmed. It involves various mathematical techniques to develop models that enable machines to recognize patterns and make decisions based on data.

Machine Learning generally involves 4 steps:

1. Data Collection and Preparation

This is probably the most time-consuming phase. It involves cleaning the data, filling in any missing data, standardising variables and converting categorical data to numerical data.

• Model Building

The clean dataset is then divided into 2 subsets – training data and testing data. The training data (typically 80% of the dataset) is then used to train the model. We define an algorithm, depending on the nature of the problem we are trying to solve, and the machine starts “learning”. Through a series of iterations, it builds a model that best fits the data.

• Model Evaluation

We the use the test dataset (the other 20%) to evaluate the model and assess its predictive ability. Depending on how it performs, we might return to step 2, try a different algorithm or optimisation function, tweak the hyperparameters and re-train. (We may repeat this loop several times).

• Deployment and Monitoring

Once we are happy with the model, it can be deployed to make real-time decisions on new data.

There are several broad categories of ML:

Supervised Learning involves training a model on a labelled dataset, meaning that the dataset has predefined outputs for each input. The goal is for the model to learn to accurately predict the output for new, unseen inputs.

For example, you want to build a model to predict house prices. You have a dataset of houses sold in the area that consists of various features – type of property, number of rooms, area, etc – as well as price achieved at sale.

You train the model on the dataset that includes the prices (the labels) and then deploy it on new property data in order to predict the price.

Or you train an image classification model on images labelled ‘dog’, ‘cat’, ‘horse’ so that when you show it an image of an animal it can predict whether it is a dog, cat or horse.

In addition to image classification, supervised machine learning can be applied to credit decisioning, spam filtering and fraud detection.

Unsupervised Learning involves training a model on an unlabelled dataset, where the inputs do not have predefined outputs. The goal is for the model to identify patterns or groupings within the data without any guidance. Some common use cases for unsupervised machine learning include:

Clustering – identifying groups of similar data points within a dataset, such as grouping customers based on their purchasing behaviour – useful for creating segmented marketing campaigns.

Anomaly detection – identifying unusual or rare occurrences within a dataset, such as detecting unusual patterns in network traffic that could indicate a cyber-attack.

Dimensionality reduction – reducing the number of features in a dataset while preserving the most important information, such as reducing the number of variables needed to predict a certain outcome.

Reinforcement learning is a type of machine learning where “the machine learns” to make decisions in an environment by interacting with it and receiving rewards or punishments. The goal is to learn a policy or a set of actions that maximizes the cumulative reward over time.

The key difference between reinforcement learning and other types of machine learning is that the machine is not given labelled data or specific instructions on how to act. Instead, it learns by trial and error, through a process of exploration and exploitation, gradually discovering which actions are more likely to lead to rewards.

Reinforcement learning has been applied to a wide range of problems. Some examples of reinforcement learning applications include teaching a computer program to play a game of chess or to control a robot arm to perform a specific task. Reinforcement learning is also used to train autonomous vehicles to make decisions in real-world driving scenarios.

Obviously, this is a fairly simplified overview. Feel free to reach out to me if you’d like to find out more or discuss a specific use case.