If you’ve ever asked how does machine learning work, here’s the short answer: a computer studies past examples, finds patterns, then uses them to predict new cases. You don’t need a maths background to follow it.
This guide sits within the wider topic of machine learning basics, and it walks through the process one step at a time.
| Item | Value |
|---|---|
| Year Arthur Samuel wrote his checkers program | 1959 |
| Point when that program reached respectable amateur level | Mid-1970s |
| Stages in the basic ML workflow | 5 stages |
| Size of the worked-example data set | 500 students |
| Part of that data used for training | 400 students |
| Part held back for validation | 100 students |
The figures come from a chapter of Real-World Machine Learning and a university course text. Figures can change, so check the source before you rely on them.
Machine learning versus traditional programming
In traditional programming, a person supplies every explicit rule to the computer. In machine learning, the computer learns the rules for itself from historical data, with a few hints from people, as a Harvard-hosted teaching page explains.
This matters because nobody has to think up every possible input in advance. The trained model makes best guesses on its own, using rules it created during training.
The idea is not new. In 1959, computer scientist Arthur Samuel wrote a program to play checkers, and he had it play many games against itself to refine how it scored board positions. By the mid-1970s the program had reached the level of a respectable amateur player, according to the Real-World Machine Learning chapter. It improved its own performance through experience.
Four terms worth knowing
A few words come up in every explanation of machine learning, so it helps to meet them early:
- Instance (or example): a single object, observation, transaction or record.
- Features: the input attributes used to predict the target.
- Target (or label): the attribute of interest, the thing to be predicted for each new instance.
- Training data: the set of instances with a known target, used to fit the model.
Put simply, a model is a mathematical object that describes the relationship between the features and the target.
The workflow from data to prediction
Most projects follow the same arc, which the Real-World Machine Learning glossary describes as the ML workflow. It has five stages, listed here in order.
- Data preparation. Raw data is converted into formats the model can accept, and the inputs are structured so the important features stand out.
- Model building. The algorithm is given the training data and learns the relationship between the features and the outcome.
- Evaluation. The model is checked on examples it has not seen, to find out whether its predictions are accurate.
- Optimization. The model is refined, and the team may choose between several competing results.
- Prediction. The finished model is used to predict the target for new cases.
The clever part of machine learning is usually the training phase, not the mathematics. A course text from La Trobe University compares training to the way humans develop experience through exposure to stimuli.
The Harvard-hosted page puts it in everyday terms: you give the model practice problems, then let it guess and check, learning and improving along the way. That is very similar to how you might study for a test.
Preparing the data
Good preparation takes real care. It can mean converting variables into more appropriate formats, checking for samples that could have excessive influence on the model, and sometimes removing variables that are highly correlated with others.
Some models also need inputs flattened or filtered so that the features they should look at are easy to pick out. If the features are poorly defined or chosen, the resulting model may not be very accurate.
A worked example: predicting exam results
The La Trobe text uses a simple scenario: predicting future students’ final grades for a statistics subject. To do it, you would collect data from current students, and each question you ask becomes a feature variable.
- The average time they spend studying each day
- Their marks for a previous statistics subject
- Their sleep patterns
- Their diet
- Their exercise routines
The exam score you want to predict is the outcome variable. Once the students sit the exam, the feature values and the outcome values together form the data set.
Classification or regression?
The kind of outcome decides the kind of problem. If the outcome is a grade such as A, B or C, it is a classification problem, and reducing it to pass or fail makes it a binary classification problem.
If you instead predict an overall mark out of 100, the outputs are continuous, and the problem becomes regression.
How training and validation data are split
Once preparation is finished, the data is typically split in two. The larger portion becomes the training data, and the remainder is kept back as validation data.
In the course text’s example, the data set holds information from 500 students. Of these, 400 are randomly selected to train the model, and the remaining 100 are used to validate it, because the machine has not seen them before, as the course text describes.
If the results on the validation data are good, the model can be used to predict the grades of future students. You would survey them at the start of the semester and feed their answers to the model.
The text also notes that, generally speaking, more training data means a more accurate model. That is why the training portion is much larger than the validation portion.
Why data quality decides the result
A model is only as good as its training data. The Harvard-hosted page says the data must be clean, well managed, labelled correctly and drawn from a diverse background, meaning many different inputs from many different people.
Otherwise the system could show bias, which is unfair to some people. The page also warns about overtraining, where a model becomes good only at identifying the inputs it has already seen and nothing else.
That is why a model should be accurate on new data and not just on its training data. After all, with supervised learning you want accurate predictions about the future.
The course text sums up the limits in one phrase: garbage in, garbage out. It also reminds readers that the patterns machine learning finds are not laws of nature. They are simply fitted curves, and if the data is noisy, the curves will not fit well.
Supervised and unsupervised learning
The course text names two main types of machine learning: supervised and unsupervised. In supervised learning, you train an algorithm on labelled data with a clear outcome variable, so it can classify or predict outcomes.
In unsupervised learning, the data has only features and no clear outcome. The aim is to search for hidden patterns, using problem classes such as clustering, dimensionality reduction and association analysis.
The exam example above is supervised, because every student has a known result. If you want to compare the two approaches in more depth, the guide on supervised vs unsupervised learning is the next step.
This page stops at prediction. Keeping a model running and reliable afterwards is a separate discipline, which the explainer on what is mlops covers.
Frequently asked questions
Is machine learning the same as AI?
Show answer
Not quite. The Harvard-hosted teaching page describes AI as a set of techniques computers use to act on their own, and machine learning as a step further, where computers learn rules from historical data and then make predictions.
Does more data always make a better model?
Show answer
Generally speaking, more training data gives a more accurate model, according to the La Trobe course text. Quality still matters: data that is unclean, mislabelled or not diverse can lead to bias and poor predictions.
What is overtraining?
Show answer
Overtraining happens when a model becomes good only at identifying inputs it has already seen. That is why part of the data is kept back to test the model on examples it has not met.
What is a feature in machine learning?
Show answer
A feature is an input attribute used to predict the target. In the exam example, study time, sleep patterns and previous marks are all features.
