An introduction to the colourful cast of characters¶

Before a model can be born it has to be some particular kind of model, and the kinds make up a mismatched football squad playing to Ankh-Morpork rules, which is to say the rules are negotiable and the river is in play. There are the by-the-book defenders, supervised learning, doing what they are told with the rigid enthusiasm of a city watchman counting barrels. There are the chaotic strikers, unsupervised learning, darting about with no clear plan and occasionally putting the ball in the water. There are the ones who borrow another squad’s training and pass it off as their own, transfer learners, and the ones who will only move if somebody keeps confirming they are doing it correctly, active learners. And there is always the player who scores own goals with remarkable consistency, otherwise known as reinforcement learning on a bad day.

Between them this squad routes traffic, decides who gets a loan, guesses which patients will next arrive at the hospital doors, sorts cat photographs from dog photographs, recommends dreadful television, and occasionally, when properly motivated, spots a tumour. Our model does not yet know which of them it will turn out to be, so it is worth meeting the relatives before the birth.

Paradigms¶

Working code for these lives in the ml and deep-learning repositories, as notebooks, preprocessing pipelines and evaluation examples, for anyone who would rather read source than prose.

Supervised learning¶

Supervised learning is like teaching a child with flashcards, except the child is an algorithm and the flashcards are thousands of labelled data points. Show it pictures of cats and dogs, explain which is which, and hope it does not conclude that every four-legged creature is probably a moggy. The task is to find patterns mapping inputs to outputs that hold up on examples it has never seen. It fails in more than one way. It can memorise the training set instead of learning from it, which shows up as a model that scores well on data it has met and badly on data it has not. Or the training set can have left a hole: show it ten thousand furry cats and it will meet its first Sphynx with no opinion worth having. Regularisation helps the first and cannot reach the second. That one needs different cats, not more of them.

It earns its keep wherever historical examples carry known outcomes and nobody fancies reviewing the next ten thousand by hand: spam filtering, credit scoring, fraud detection, medical diagnosis, churn prediction, insurance risk assessment.

The spam filter is the classic. Trained on millions of emails marked “spam” or “not spam”, it learns that “free” and “Nigerian prince” are red flags, then files an urgent message under junk because it contained the phrase “meet me tomorrow”, while the actual spam about dubious pills sails through on the grounds that it looks like a medical newsletter.

There are privacy and security concerns too. Where the training data holds sensitive information, a clever attacker may be able to work backwards from the model’s outputs. A health insurance predictor might let slip that people aged 30 to 40 who buy gluten-free pasta are high-risk. The model can also be poisoned with fake labels, turning a loyal spam filter into a willing accomplice for scam emails.

Unsupervised learning¶

Unsupervised learning is the algorithm equivalent of dumping a pile of puzzle pieces in the middle of the Guildhall and saying, “Sort that out.” No guidance, no labels, just raw data and existential dread. The algorithm clusters similar items, simplifies data, or flags odd outliers. Think of it as a detective trying to solve a crime without knowing there has been a crime, only that something statistically unusual has occurred.

Customer segmentation, anomaly detection in financial transactions, market basket analysis, content discovery in recommendation systems. Anywhere data has piled up and nobody has got round to labelling it, which is most places.

Netflix groups viewers into taste clusters, which is how someone ends up three episodes into a true crime series at one in the morning while the sidebar gently suggests a documentary about sleep. Card issuers use the same machinery for fraud detection, which is why a single banana bought in Spain can freeze an account. Privacy risks abound, particularly when rare behaviour patterns reveal sensitive information, such as purchases indicating pregnancy or a medical condition.

Transfer learning¶

Transfer learning is standing on the shoulders of giants, where the giants are algorithms that did the hard work first. A model trained to recognise cats can be repurposed to spot tumours. The knowledge accumulated during one task is applied to a different but related problem, saving considerable time and compute that would otherwise be spent learning that edges exist.

Medical imaging, manufacturing quality control, document classification: the pattern is a domain starved of labels sitting next to a related domain that is not. Materials scientists use it to screen candidate materials, and radiologists to read scans with networks that learned their first edges on holiday photographs.

The risks arrive with the inheritance. Hidden biases in the pre-trained model come along uninvited into the new domain, and a fine-tuned model can itself be reverse-engineered from its outputs by someone who decides the competitive advantage is now theirs.

Neural network architectures¶

Architecture is the shape of suffering settled on once the decision to use a neural network has already been made. Feedforward networks are the baseline, handling tabular classification and regression: customer churn, loan default, medical risk scoring. Convolutional networks take image and signal data, powering satellite imagery analysis and quality inspection on factory floors. Recurrent networks and LSTMs process sequences, and dominated log anomaly detection from 2017 to about 2021. In retail demand forecasting gradient-boosted trees took the podium. The M4 forecasting competition was won outright by an LSTM hybrid, with a gradient-boosted meta-learner second. Generative adversarial networks produce synthetic data for drug discovery, fraud simulation, and creative work that regularly unsettles people in marketing meetings.

Then there are transformers. The trick is attention: instead of walking a sequence one step at a time and hoping the beginning survives to the end, the model weighs the positions available to it and learns which ones bear on which. Classical recurrence never scaled that way, since each step waited on the last. Almost every large language model in use is built on this, which is why prompt injection is something anyone has to think about at all. Attention has a cost, though: weighing positions against one another means work that grows with the square of the sequence length. State-space models are a standing challenge on that point, trading that comparison for something nearer linear, and so far they have taken a share of the long-context work rather than the field.

A deep network does not explain itself by construction, and what can be recovered afterwards by attribution and probing methods is partial and disputed. A wrong answer generally gets caught by its consequences.

Further paradigms¶

The territory runs well past the edge of any one codebase, and the rest of the family turns up in production whether or not anyone has written code for it here.

Semi-supervised learning¶

Semi-supervised learning is the lazy student of the class, working from a few labelled examples and a mountain of unlabelled data. The labelled handful act as stabilisers on a bicycle already going downhill rather faster than anyone intended. Google Photos does this: label a few pictures of Karen and the algorithm cheerfully tags the golden retriever as Karen too. Speech recognition uses it as well, which is how “call Mum” becomes “ball bomb”.

Content moderation leans on it, since labelled examples of harmful content are expensive to produce in every sense of the word. So do web page classification and medical record analysis, where annotation costs a clinician’s time and clinicians are not sitting idle.

Risks include the unlabelled data containing private information and incorrect labels propagating errors. Imagine a semi-supervised hate speech detector that learns from bad examples and subsequently flags all political discourse as toxic.

Reinforcement learning¶

Reinforcement learning is like training a dog with treats, except the dog is a robot and the treats are mathematical rewards. The agent tries actions, receives feedback, and adjusts. AlphaGo Zero is the pure case, learning Go from nothing but the rules and several million games against itself, in the manner of a sleepless prodigy with no friends. Its predecessor AlphaGo had a gentler upbringing, starting from a network trained on human expert games before it was set loose on itself.

Algorithmic trading, robotic process automation, supply chain optimisation, autonomous navigation, personalised pricing. The common thread is sequential decisions taken in an environment that answers back, with nobody on hand to label the optimal move at each step.

Reward shaping is where it goes wrong: specify the reward carelessly and the agent will find the cheapest way to collect it, which is rarely the behaviour anyone had in mind. These systems can also leak what they learned about the environment they inhabit, the layout of a house included.

Ensemble learning¶

Ensemble learning is asking a panel of slightly drunk pundits, averaging the answers, and calling the result wisdom. It works, and it works in two distinct ways that often get lumped together. Bagging, the method behind a random forest, trains many high-variance models on different samples and averages out their disagreements. Boosting, the method behind gradient-boosted trees, trains models in sequence, each attending to what its predecessor got wrong. The first reduces variance, the second reduces bias, and stacking hands the whole argument to a further model trained to weigh the opinions.

Insurance underwriting, clinical decision support, financial risk modelling, and competitive machine learning, where the last fraction of a percentage point is worth the trouble.

The structure changes the exposure, though less tidily than the folklore suggests. A single corrupted learner in a voting ensemble mostly gets outvoted, and there are aggregation schemes built deliberately around that property, training each member on a small disjoint slice so that a bounded number of poisoned points can reach only a bounded number of members. An off-the-shelf random forest is not one of those: bootstrap sampling puts any given point into most of the bags. Sequential methods have a different difficulty, since each learner is fitted against whatever the last one left behind, and AdaBoost in particular piles weight onto the examples it keeps getting wrong, poisoned or otherwise. None of the arithmetic helps once the corrupted points stop being few.

Federated learning¶

Federated learning is the Ankh-Morpork neighbourhood watch of algorithms. A device learns from local behaviour and sends the lessons rather than the data to a central server. The raw data really does stay put, which is not a marketing fiction. What travels is the update, and the update turns out to be the interesting part.

Mobile keyboard prediction; fraud detection across banks that cannot legally pool customer records; healthcare research across hospitals under different data protection regimes. The regulatory appeal is obvious, the technical complexity considerable, and the gap between the two is where most projects quietly expire.

Gradients carry a good deal more about their training data than intuition suggests, and a determined observer on the receiving end can reconstruct some of it. Protocols exist for aggregating updates so that individual ones are hidden, and they are partial measures.

Three ways of reasoning¶

Inductive learning¶

Inductive learning generalises from examples, which is both the point of it and the hazard. A traffic warden tickets one car and concludes the whole street is at it. Credit scoring, insurance risk assessment and customer lifetime value prediction all work this way: past behaviour pressed into service as a guide to future behaviour, with the confidence of someone who has never met an edge case.

Adversarial manipulation of data can fool the system, and sensitive correlations may be revealed without anyone intending to reveal them.

Deductive learning¶

Deductive learning reasons from rules already held. Where it reads examples at all, it discovers nothing the rules did not already entail; what it usually gains is speed, by compiling a laborious proof into something it can reach for next time. Tax software runs on rules and no examples whatever, and fails whenever reality declines to comply. It suits regulatory compliance checking, rules-based fraud detection, and any domain where auditability counts for more than flexibility and somebody has actually written the rules down.

Privacy risks are low, but biases may be embedded in the rules themselves by whoever wrote them on a wet Thursday afternoon.

Transductive learning¶

Transductive learning skips the general rule. Rather than fitting a function that will work on whatever turns up later, it reasons straight from the labelled examples to a specific batch of unlabelled points that are already present when training starts. Less a lesson learned than one exam paper answered. Nothing survives the exercise that a new point could be handed to. The whole thing runs again. Drug interaction prediction uses it where the compounds of interest are all present from the start and the question is which pairs interact. Few-shot work uses it too, where the classes are new but the points to be labelled are all in hand.

Privacy risk is high, since the labelled set has to be kept and consulted rather than distilled away into weights.

Human-in-the-loop models¶

Reinforcement learning from human feedback¶

Where no reward function can be written down, one gets learned instead. Reviewers compare outputs in pairs, a reward model is fitted to which they preferred, and the policy is then optimised against that stand-in rather than against anything anybody stated. Large language model alignment runs on this, because what people want from an answer turns out to be remarkably hard to put into words and comparatively easy to point at.

Summarisation was where the technique was first put to work, and it lives now in dialogue assistants and in the preference tuning of image generators.

The labour behind it is real and largely invisible: sizeable teams of reviewers, frequently underpaid and rarely credited, teaching a model which of its answers to keep to itself. Feedback channels can also be gamed.

Active learning¶

Active learning is the needy intern of algorithms, constantly asking “is this right?” Rather than demanding that everything be annotated before it will condescend to learn anything, it picks which points to ask about and sends those to a human. How it picks is where the field lives. It might ask about the ones it is least certain about, the ones a committee of models disagree over, or the ones expected to shift it most.

Medical imaging analysis and legal document review, wherever labelling is expensive and expert time is the binding constraint. The queries themselves can reveal sensitive information, and blind spots form in the regions the model never thinks to ask about, which is reliably where the interesting failures live.

Whatever kind it turns out to be, our model does not get a vote in the matter. Meeting the family was the easy part.