01 · How learning works 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Think about learning with a teacher Examples support practice. Feedback helps us improve. New questions check whether we can use what we learned.
Introduce four roles, not three stages or rules. Study material supplies examples. The student learns a method through practice. The teacher provides feedback. Assessment uses new questions to check whether learning transfers. 01 · How learning works 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Machine learning has four corresponding roles Study material Data
Examples and measurements.
Learner Model
A fixed structure; adjustable settings.
Teacher Training algorithm
A procedure for improving the settings.
Assessment Evaluation
Check on examples kept out of training.
Data quality matters. Training updates the model’s adjustable settings (parameters).
The model has a fixed structure and adjustable parameters. The training algorithm changes the parameters using the data and objective; the evaluator measures performance. Data quality matters. Held-out means not used for that training step. 01 · How learning works 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Can examples help predict the next harvest? Earlier field trials: record fertilizer amount and the harvest size (yield). Estimate a harvest before measuring it Input: fertilizer amount for a field.
Output: predicted harvest size (yield).
Past field trials provide examples where both are already known.
Past examples Fertilizer + measured yield
→ Learn a relationship Fit the model to those examples
→ Predict a new harvest Use the new field’s fertilizer amount
A field trial means growing a crop with a recorded fertilizer amount, then measuring its harvest.
Use field and field trial rather than plot, which could mean a chart. Each historical field trial supplies an input and measured output. Fit a prediction model from those pairs, then estimate yield for a new field from its fertilizer amount. This is an association, not a causal agricultural recommendation. 01 · How learning works 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Learning a rule and using it are different steps Training Examples
→ Adjust model
→ Fitted model
Parameters change as the model learns.
Inference New input
→ Fitted model
→ Prediction
Parameters stay fixed during this prediction.
Training updates parameters; inference applies the fitted model to an input.
In the crop example, changing parameters from fertilizer/yield examples is training. Predicting yield for a new field with the fitted parameters is inference. Online systems can alternate these stages. 01 · How learning works 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications The key question: where does feedback come from? Supplied answers Input + reference target
Supervised
The observations Find patterns without task labels
Unsupervised
Consequences of actions Act, observe a reward, improve
Reinforcement
The boundaries overlap: some methods combine labels, derive targets or use several learning stages.
These are broad families, not disjoint boxes. Semi-supervised methods combine labeled and unlabeled examples. Self-supervised methods derive targets from observations. Both receive their own place in the chapter map. 02 · Supervised learning 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Supervised learning: compare with a known answer Fertilizer level → measured yield Animal image → supplied label “dog” Make a prediction
→ Compare with the target
→ Update the model
Prediction: ŷ = fθ (x).Ground truth: the reference answer or measurement; it can contain errors.
Both examples are supervised because a reference target is available during training. Introduce ground truth as the reference measurement or annotation, not a promise that it is error-free. The function is f, the parameters are θ, x is input, y is target and ŷ is prediction. 02 · Supervised learning / 2.1 · Regression 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Regression predicts a numerical magnitude Input: fertilizer level Output: estimated yield A value on a numerical scale, such as yield, temperature or travel time.
Regression predicts a magnitude. Differences between its numerical output values have quantitative meaning.
The output scale is illustrative: 3.2 kg shows the format of a quantitative prediction. The photograph is context, not its measured source. The original 25 observations and fitted-curve illustration are explored in Part 2. 02 · Supervised learning / 2.2 · Classification 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Classification predicts one of the named categories Input: an animal image Output: a class label { cat, dog }
Image
→ Classifier
→ Predicted class
Reveal the recorded label cat
This supplied answer is the training reference.
Labels are supplied during training. For a new image, the model must predict the label.
Reveal the reference label after students identify the output type. This is a photo-based concept check, not a running classifier. The target categories are known in advance. A numeric code for a category still represents a class. 02 · Supervised learning / 2.3 · Compare the targets 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications The target’s meaning decides the task 2.1 · Regression A magnitude yield: 3.2 kg
Differences between values have quantitative meaning.
2.2 · Classification A category species: dog
Codes 0, 1, 2 can name categories too.
Ask what the target means: category codes are labels, not measurements.
The kilogram value is an invented output-format example, separate from the source chart whose physical units are unspecified. Numerical storage alone does not decide whether a target is quantitative. 02 · Supervised learning / 2.4 · The cost of labels 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Useful examples need useful labels label: ? An image is available. Is the right annotation available too? Labeling can be the bottleneck Someone may need to identify an object, draw its boundary or measure an outcome.
Many inputs
→ Fewer reliable targets
Next: what can we learn without these task labels?
Missing or expensive task labels motivate learning from unlabeled observations.
Labels may be produced by people, records or instruments. Cost, expertise and measurement delays can limit the quantity and quality of labels. This motivates the unsupervised and self-supervised chapters; it does not imply that their training is always cheaper. 03 · Unsupervised learning 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Unsupervised learning: start without task labels What can we learn? Observations
→ Useful structure
Clustering: group similar observations.
Generative modeling: learn patterns to produce new examples.
Without task labels does not mean without a training objective.
The photographs have no supplied task labels in this illustrative pool. The following two slides separate grouping from generation. Unsupervised learning still requires objectives, representations and choices about useful structure. 03 · Unsupervised learning / 3.1 · Clustering 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Clustering groups similar observations Choose features to compare observations. Similarity depends on those features; clusters need not match human categories.
Each dot stands for one observation in a chosen representation. Distance expresses a chosen similarity. These are schematic points, not features computed from the photographs. Clusters need not correspond to species or other human categories. 03 · Unsupervised learning / 3.2 · Generative modeling 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Generative modeling learns patterns for new samples Generate a new sample with learned patterns; assigning a label is a different task.
Use the varied roof and size to distinguish generating a new sample from assigning a class. These drawings are a schematic illustration, not outputs of a running model. Generative modeling need not reconstruct every input or simply copy a training example. 03 · Unsupervised learning / 3.3 · Self-supervised learning 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Self-supervised learning: create a target from the data ? Training input: visible pixels, with a region hidden. The original supplies the answer Hide part of an observation
→ Predict the hidden part
Compare the prediction with the content you withheld.
Next-token prediction:
Tokens are units of text, such as words or word parts. Self-supervision derives its targets from recorded observations.
A mask is an overlay on the source photograph. The withheld original content supplies a target without human task annotation. The next interaction constructs the same idea with text. A token may be a word or part of a word. 03 · Unsupervised learning / 3.3.2 · Pretraining and adaptation 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Learn broadly first; adapt to a task next Unlabeled observations Task-specific examples Pretraining Learn reusable patterns
→ Fine-tuning Continue training for the task
→ New task Evaluate the adapted model
Fine-tuning continues training on the target task; successful transfer still needs evaluation.
Pretraining can use self-supervised targets from a large unlabeled pool. Fine-tuning continues training on task-specific examples, often with fewer labels than training from scratch. Transfer depends on relevance to the new task and still requires evaluation. 04 · Semi-supervised learning 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Few labels, many examples: semi-supervised learning Labeling every photo takes time. Can the unlabeled photos help too?
100 labeled photos: image + species + 10,000 unlabeled photos: image only → Train together Use labels and patterns in both pools
Semi-supervised learning trains on labeled and unlabeled examples for the same task. Unlabeled data helps when its patterns are relevant to the labels.
Explain the motivation and define the training-data mixture before naming prediction goals. The counts and photo labels are illustrative. Semi-supervised methods use both supplied targets and information from unlabeled observations; improvement is not guaranteed. 04 · Semi-supervised learning / 4.1 · Prediction goals 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Two goals: label this batch, or label future photos Start with 100 labeled photos and 10,000 unlabeled photos.
Transductive learning These 10,000 photos → Labels for this batch
The objective of the learning is to classify the unlabeled data only, without the necessity of generalizing over unseen data.
Inductive learning Learn a classifier → Label tomorrow’s photos
The objective is to teach the model to label also unobserved data.
Semi-supervised describes the training data. Transductive and inductive describe which cases we want to predict.
Transduction targets the particular unlabeled batch supplied now. Induction learns a function to apply to new inputs beyond that batch. Induction is not defined as also labeling the batch. Both goals can use a labeled/unlabeled mixture. These are goals, not guarantees that a method has succeeded. 04 · Semi-supervised learning / 4.2 · Weak supervision 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Weak supervision: learn from imperfect labels Accurate, detailed labels can be costly. We may have only partial, coarse or unreliable answers.
Incomplete ? Some examples have no label.
Coarse dog present The label does not locate the dog.
Noisy cat ✕ The supplied label can be wrong.
Weak supervision uses incomplete, coarse or noisy signals. Semi-supervised learning addresses the incomplete-label case by also using unlabeled data.
The same photograph is repeated with deliberately different teaching annotations. Weak supervision covers incomplete, inexact and inaccurate signals. Domain shift is a different issue and follows next. 04 · Semi-supervised learning / 4.3 · Domain shift 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications The same task can arrive in different-looking data Training appearance · website A Deployment appearance · website B (simulated) Changed lighting / compression
→ Adapt and evaluate on the target domain
Domain shift changes the input distribution. Weak supervision describes label quality.
The same original photo is displayed twice, with a darker display on the right. It illustrates acquisition change rather than depicting actual website A/B datasets. Adaptation can involve normalization or representation alignment. New target classes are an additional problem, not just changed lighting. 05 · Reinforcement learning 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Reinforcement learning: improve by acting A move changes what happens next. Feedback comes from consequences The goal is accumulated reward. A useful reward may arrive several moves later.
Policy = action-selection rule · reward = outcome feedback · state = current situation.
Chess motivates learning from interaction instead of a correct move label for each state. State is the current situation; policy is an action-selection rule; reward evaluates an outcome. RL can also use previously collected experience. 05 · Reinforcement learning / 5.1 · Exploration and exploitation 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Use what you know—or try what you do not? ε-greedy Explore with probability ε
→ Otherwise exploit with probability 1 − ε
ε (epsilon) = exploration probability. An exploratory choice need not give a higher immediate reward.
Epsilon is the exploration probability. Current reward estimates come from observations and may be wrong. Exploration buys information, not guaranteed immediate improvement. The next bandit isolates this tradeoff without a changing board state. 05 · Reinforcement learning / 5.2 · Bandit setup 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications What is a bandit arm? A bandit problem: repeatedly choose an action and observe its uncertain reward.
One round 1. Choose an arm: select action A, B or C.
2. Observe its reward: in our example, 1 or 0.
3. Update its estimate: use what that choice taught you.
An arm is an action; a pull is choosing it once. The name comes from a slot-machine lever.
You see only the chosen arm’s reward. The same arm can give 1 now and 0 next time. The goal is to maximize total reward across many rounds.
The slot-machine picture explains the terminology, not a gambling activity. In the demonstration, each arm has a fixed but unknown probability of reward 1. Outcomes vary from pull to pull. Only the chosen outcome is observed. The learner estimates each rate from its own observations and balances immediate reward against information. Unlike chess, this bandit has no changing board state. 06 · Application domains 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications The learning setting and the data domain are different Computer vision Images and video.
Natural language The crop grows.
Text and token sequences.
Multimodal “A cat walking on snow.”
Combine image and text.
Ask two questions: what kind of data, and what kind of training feedback?
A domain says what data/task we work with. A learning setting says how feedback is obtained. A vision system could be supervised, self-supervised or use several stages. 06 · Application domains / 6.2 · Natural language 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Context changes both a word’s role and its meaning Part-of-speech tagging They bank the aircraft.
bank → verb Word-sense disambiguation the river bank
bank → edge of a river “A late goal decided the final.”
→ Document topic → sports
Document classification, grammatical tagging and word meaning ask different questions.
Part-of-speech tagging identifies grammatical roles. Word-sense disambiguation identifies intended meaning. Document topic classification predicts a document-level category. Tokens may be words or parts of words. 06 · Application domains / 6.3 · Multimodal learning 1 · Learning 2 · Supervised 3 · Unsupervised 4 · Semi-supervised 5 · Reinforcement 6 · Applications Multimodal learning connects different kinds of data Image input Text input A cat walking on snow.
Image + text
→ Learn their relationship
Example task: match an image with the appropriate caption.
Image/text and audio/video are data combinations; they do not specify the learning setting.
Multimodal means combining data types, such as image/text or audio/video. Matching an image and caption illustrates a cross-modal relation. The data domain alone does not decide the learning signal. 
Past examplesFertilizer + measured yield
Learn a relationshipFit the model to those examples
Predict a new harvestUse the new field’s fertilizer amount
