Supervised vs Unsupervised vs Reinforcement Learning, With Examples

Almost every introduction to machine learning starts by splitting it into three types: supervised, unsupervised, and reinforcement learning. The map is useful, but the textbook definitions are easy to forget, so each type below comes with concrete examples.

The cleanest way to tell them apart is to ask what kind of feedback the algorithm gets. Supervised learning is handed the right answers. Unsupervised learning gets no answers at all and has to find structure on its own. Reinforcement learning gets no answers either, but it does get rewards and punishments as it experiments.

The three at a glance

Supervised learning: learning from answers

In supervised learning, every training example comes with a label, which is the correct answer. You show the model thousands of emails already marked spam or not spam, and it learns the pattern that separates the two. Show it enough labelled photos of cats and dogs, and it learns to tell a new one apart. This is the most widely used type in practice, largely because so many real business problems boil down to “predict this from that.” The main cost is the labels themselves: someone, somewhere, had to tag all that data, and good labels are expensive.

Supervised problems come in two main shapes. Classification predicts a category, such as spam or not spam, or which of ten digits is in an image. Regression predicts a number, such as next month’s sales or the price of a house. A wide range of algorithms handle both, from linear and logistic regression to decision trees, random forests, gradient boosting and neural networks (see our explainer on artificial neural networks). Whatever the algorithm, the workflow is similar: split the data into training and test sets, fit the model on the first, and judge it only on the second, so you measure how it handles examples it has never seen.

Take the spam filter. The features might be the words in the message, the sender and the links it contains, and the labels might come from users who marked messages as spam. The model learns which combinations of features tend to go with which label. Evaluation needs care, though. If only 1% of messages are spam, a filter that never flags anything is 99% accurate and completely useless, which is why practitioners look at precision (how many flagged messages really were spam) and recall (how much of the real spam was caught). The scikit-learn guide to supervised learning walks through the standard algorithms.

Unsupervised learning: finding structure in the dark

Unsupervised learning gets no labels. You hand the algorithm a pile of data and ask it to find the structure hiding inside. The classic job is clustering: grouping customers by behaviour without anyone telling the model what the groups should be. It is also used to spot anomalies (a transaction that does not resemble the others) and to compress data down to its essentials. Because there is no answer key, evaluating an unsupervised model is harder. There is no single correct answer, so you are often judging whether the patterns it found are useful.

Unsupervised methods come in a few main families. Clustering algorithms such as k-means or hierarchical clustering group similar items together. Dimensionality reduction techniques such as principal component analysis (PCA) squeeze many variables into a few that preserve most of the variation, which makes the data easier to plot and process. Anomaly detection flags items that look unlike the rest. A retailer might cluster shoppers by what they buy and how often, then discover a group of frequent small-basket buyers that nobody had named. The algorithm only finds the grouping. A person still has to decide what it means and whether acting on it makes sense. The scikit-learn documentation on unsupervised learning is a good practical reference.

Reinforcement learning: learning from consequences

Reinforcement learning works differently from the other two. An agent takes actions in an environment, and instead of being told the right move, it receives rewards or penalties and gradually works out a strategy that earns the most reward over time. It is how a program learns to play chess or Go at superhuman level, and how robots learn to walk. The trade-off is that it can be slow and data-hungry, since the agent often has to try a great many bad moves before it finds good ones.

A few terms make reinforcement learning much easier to read about. The agent is the learner, the environment is whatever it acts on, a state is what the agent currently observes, and the policy is its rule for choosing an action. The reward is the score that comes back. Two challenges define the field. The first is exploration versus exploitation: should the agent repeat the move that has worked so far, or try something new that might work better? The second is delayed reward. In a game of chess the payoff only arrives at the end, so the agent has to work out which of dozens of earlier moves deserves the credit or the blame.

In 2015 a DeepMind team showed that a single deep reinforcement learning agent could learn to play many Atari games from raw screen pixels and the game score alone, reaching a level comparable to a professional human tester on many of them (Mnih et al.). In 2016 the AlphaGo team described a system combining deep neural networks with tree search that beat a professional Go player, and AlphaGo went on to defeat the world champion Lee Sedol 4 to 1. These successes all happened in settings with a cheap simulator and a clear score. Outside games, designing a reward that captures what you actually want is hard, and an agent that practises in the real world can be slow, costly or unsafe. For a full treatment, Sutton and Barto’s textbook is freely available online.

Where the lines blur: RLHF

These categories are a map, not a wall, and modern systems happily mix them. A good example is how large language models are fine-tuned. Reinforcement learning from human feedback (RLHF) uses human preference ratings as a reward signal to steer a model toward more helpful, less harmful responses. It is a practical use of reinforcement learning, and a large part of why today’s AI assistants are as usable as they are.

The idea has a clear lineage. Christiano and colleagues showed in 2017 that an agent can learn complex behaviour from human preferences between pairs of its own attempts, without anyone writing down a reward function, while needing feedback on less than one percent of its interactions. OpenAI’s InstructGPT work (Ouyang et al., 2022) applied the recipe to language models in three steps: fine-tune a model on human-written demonstrations (supervised learning), train a model of human rankings of its outputs, then use reinforcement learning to push the model toward the answers people rank higher. In their human evaluations, outputs from a 1.3 billion parameter InstructGPT model were preferred over those of the 175 billion parameter GPT-3, despite having about 100 times fewer parameters. All three learning types show up in a single pipeline. If you want to compare RLHF with newer approaches, our guide to RLHF, Constitutional AI and RLAIF continues from here.

If you are mapping out how these learning methods feed into actual model architectures, our ANN vs CNN vs RNN vs GNN cheat sheet is a useful next stop.

Other learning setups

Real projects rarely fit neatly into three boxes. Semi-supervised learning uses a small set of labelled examples alongside a large pile of unlabelled ones, which is useful when labelling is the expensive part. Self-supervised learning builds the labels out of the data itself: a language model, for instance, is trained to predict the next word in a sentence, so the text supplies both the question and the answer. That trick is what lets modern models learn from enormous amounts of unlabelled text. Transfer learning then takes a model trained on a broad task and fine-tunes it on a narrow one, so a team with a few thousand labelled examples can still get strong results. For how these ideas relate to deep learning and neural networks, see our guide to machine learning vs deep learning vs neural networks.

How to choose an approach

Start from the data you have and the question you need answered:

• You have labelled examples and want to predict something: supervised learning. This covers fraud flags, demand forecasts, image tagging and most business prediction problems.
• You have raw data and want to understand it: unsupervised learning. Segmenting customers, spotting unusual behaviour or compressing a large dataset for plotting are typical jobs.
• You have a goal, a way to act and a way to score the results: reinforcement learning. It suits games, robotics and control problems where a simulator lets the agent practise safely.
• You have lots of unlabelled data and a little labelled data: semi-supervised or self-supervised learning, followed by fine-tuning.

Common mistakes to avoid

• Letting test data leak into training. If any information from the test set reaches the model during training, the reported accuracy will look better than real-world performance.
• Trusting a single accuracy number. On imbalanced problems the headline figure hides failures. Check precision and recall as well.
• Treating clusters as facts. Unsupervised methods will always return groups, even when the data has no meaningful grouping. A person has to decide whether the groups mean anything.
• Rewarding the wrong thing. A reinforcement learning agent optimizes exactly the reward it is given, which can produce behaviour that technically scores well while missing the point.
• Ignoring label quality. In supervised learning, noisy or biased labels teach the model noisy or biased habits.

Key takeaways

• The three types differ in feedback: correct answers (supervised), none (unsupervised) or rewards (reinforcement).
• Supervised learning dominates in practice because most business problems are prediction problems with labelled data.
• Modern AI systems combine methods, with self-supervised pretraining, supervised fine-tuning and reinforcement learning from human feedback often used together.

Sources

• scikit-learn documentation: Supervised learning and Unsupervised learning.
• Sutton and Barto, Reinforcement Learning: An Introduction (2nd edition).
• Mnih et al., “Human-level control through deep reinforcement learning,” Nature, 2015. doi.org/10.1038/nature14236
• Silver et al., “Mastering the game of Go with deep neural networks and tree search,” Nature, 2016. doi.org/10.1038/nature16961
• Christiano et al., “Deep reinforcement learning from human preferences,” 2017. arxiv.org/abs/1706.03741
• Ouyang et al., “Training language models to follow instructions with human feedback,” 2022. arxiv.org/abs/2203.02155