Introduction to machine learning · Part 1

Think of a student learning to solve a new kind of problem. Four roles are involved: study material supplies worked questions and solutions; the student practices and learns a method; the teacher gives feedback; an assessment checks progress using new questions.

These are four roles, not four rules or a sequence of algorithm steps. The next slide maps each role to a component of machine learning.

The learning analogy maps study material to data, the learner to the model, the teacher to the training algorithm, and an assessment to evaluation. These roles are separate: having data and a model is not yet a complete learning procedure.

Learning involves four components. Data provides observations and any available feedback. A model specifies a structure, such as a family of functions, with parameters that can be adjusted. A training algorithm chooses parameter updates from the data and objective. An evaluator measures the resulting predictions against the task’s goal.

The components have different roles. A model does not change simply because it receives a new input: training must adjust its parameters. To assess whether learning transfers beyond the fitted examples, evaluation needs suitable held-out data. We examine that separation in Part 2.

Held-out examples are kept out of the training step and used to assess the fitted model. Their purpose is to check performance beyond the examples used to adjust its parameters. Data quality also matters: unsuitable examples can teach an unsuitable prediction rule.

The running task is to predict harvest size, also called yield, from the fertilizer amount known before harvest. A field trial grows a crop with a recorded fertilizer amount and later measures the harvest. Past trials provide input–target examples because both measurements are already available.

Training fits a prediction model to those pairs. For a new field, supply its fertilizer amount to the fitted model and estimate its yield before that yield has been measured. The photograph gives context only. These illustrative measurements do not establish a causal effect or provide an agricultural recommendation.

Training adjusts model parameters using examples and a learning signal. Inference uses the fitted model to produce an output for an input. A crop model can predict yield for a new field without changing its parameters; updating the parameters after the field’s measured yield arrives is another training step.

Training and inference may occur at different times or alternate in an application. Their distinction depends on what the operation does, not whether it happens online. When describing a system, name the input, output and the parameters, and state when those parameters change.

The lecture first introduces the components of learning, then explores supervised, unsupervised, semi-supervised and reinforcement learning. Within these chapters, regression and classification are supervised tasks; clustering and generative modeling are unsupervised goals; self-supervision creates targets from observations.

The semi-supervised chapter also distinguishes related issues: weak signals and domain shift. The final chapter applies the learning settings to computer vision, language and multimodal data. The numbered chapter bar in the slides and grouped contents here follow the same sequence.

A learning setting is characterized by the feedback available for training. Supervised learning uses input–target pairs. Unsupervised learning seeks structure without task-specific target labels. Self-supervised learning constructs prediction targets from observations. Reinforcement learning uses rewards associated with actions and their consequences.

These are useful distinctions, not a rule that an entire application must receive just one name. A model can undergo self-supervised pretraining followed by supervised fine-tuning. The following slides introduce each setting before the mixed-scenario exercise asks you to distinguish them.

Supervised training uses pairs consisting of an input x and a target y. For example, the input is fertilizer level and the target is the yield measured later. The model produces a prediction ŷ = f_θ(x), where f is the prediction function and θ (theta) is its collection of adjustable parameters.

The training algorithm uses disagreement between prediction and target to adjust θ. During inference, it uses the fitted model to predict a target for a new input. Targets may come from sensors, records or human annotations. “Ground truth” names the reference used for comparison; it does not guarantee a perfect measurement.

Regression predicts a quantitative magnitude, such as crop yield, temperature or travel time. Differences between output values have quantitative meaning. The slide illustrates the output format with a yield of 3.2 kg; this is not a measurement taken from the photograph.

Part 1 uses this example to distinguish an output magnitude from a class label. Part 2 develops the dataset, prediction functions and evaluation, and inspects the numerical observations in the fertilizer example.

Classification predicts membership in a finite set of categories. In a cat-versus-dog task the possible outputs are the class labels cat and dog. During supervised training each example includes its reference label; for a new prediction case that label is unknown to the model.

The reveal shows the recorded reference label. It is separate from a model’s prediction, which can be wrong. Naming classes with codes such as 0 and 1 changes their storage representation, not the meaning of the classification task.

In regression the target is a quantitative magnitude, such as crop yield in kilograms. In classification the target is a member of a finite set of categories, such as a plant species. The distinction comes from the meaning of the output rather than its storage format.

Encoding three species as 0, 1 and 2 does not turn species identification into regression. The numerical differences between those codes do not measure biological distances. Both tasks use input–target pairs when learned in a supervised setting.

A supervised dataset needs useful targets, not just inputs. Collecting many images may be easy, while asking an expert to identify every species takes time. Other targets, such as crop yield, require a later measurement rather than manual annotation.

This motivates methods that use unlabeled observations, derive training targets from observations, or combine a small labeled set with a larger unlabeled set. These approaches reduce dependence on task labels; they do not remove the need to assess whether the learned model is useful.

Unsupervised learning works with observations without supplied task-specific targets. In clustering, the goal is to group similar observations. The result depends on the representation and the meaning of similarity: grouping images by color need not group them by species.

Generative modeling learns patterns in a data distribution so that new examples can be sampled. It need not assign a category to every input or reconstruct every observation. Generative and self-supervised methods can overlap: one term describes a modeling goal, while the other describes how a training target is obtained.

Clustering organizes observations into groups using a chosen representation and notion of similarity. Unlike supervised classification, the training examples are not supplied with the desired class labels. A photo collection might be grouped by appearance, for example coat color or body shape, without those groups being predefined breeds.

Each dot in the diagram stands for an observation in a schematic representation. The groups are a teaching illustration, not clusters computed from the accompanying photographs. Different representations or similarity measures can produce different groupings; a cluster need not correspond to a meaningful human class.

Generative modeling learns patterns in a data distribution that can support generating new samples. The output might be an image, a text sequence or another kind of observation. This differs from clustering, which groups existing observations.

The house drawings illustrate the idea: examples share some structure but vary in their details, and a new sample can preserve the structure without repeating one example exactly. These are authored drawings, not samples produced by a trained model. Generative modeling and self-supervision can overlap: one describes a modeling goal, while the other describes how training targets are obtained.

Self-supervised learning constructs input–target pairs from recorded observations. From “The crop needs water”, hide “water” to form the input “The crop needs [MASK]”; the hidden recorded word supplies the target. A plausible alternative word does not change that particular training target.

Next-token prediction uses the earlier tokens as input and the next recorded token as target. An analogous image task can withhold a patch and predict its content. A token is a unit of text used by the model, such as a word or part of a word.

Select a recorded sentence and a token to withhold. The input retains the surrounding context, while the withheld recorded token supplies the target. Reveal the target to see where the self-supervised feedback comes from.

For “The crop needs water”, hiding “water” produces a training target of “water”. Another completion could be plausible English but still differ from this recorded target. This interaction demonstrates construction of a training pair; it does not run or train a language model.

Pretraining is an initial learning stage on a broad collection of data. A self-supervised task, such as predicting withheld content, supplies feedback without task-specific annotations for every example.

The resulting model can then be adapted to a particular task. Fine-tuning means continuing training using examples for that task, often with fewer labeled examples than would be needed to train from scratch. Transfer is useful when the patterns learned in pretraining are relevant to the new task; its success still needs evaluation.

Labeling an image may need human time or specialist knowledge, while collecting additional unlabeled images may be much easier. Semi-supervised learning uses both labeled and unlabeled examples for the same task. For example, combine 100 photos with known cat/dog labels and 10,000 photos whose labels are unavailable.

The labeled examples connect the observations to the target task. Patterns in the unlabeled examples may help the learner represent or separate the data. They do not automatically supply correct labels, and using them does not guarantee an improvement. Their distribution and structure must be relevant to the task.

Suppose 100 photos have cat/dog labels and 10,000 more have no labels. A transductive goal is to assign labels to those particular 10,000 photos. The required output is a set of predictions for that available batch; a rule that handles future uploads is not required by the goal.

An inductive goal is to learn a classifier that can label photos arriving later. For example, tomorrow a user uploads a photo that was absent from training, and the learned classifier predicts cat or dog. The aim extends beyond the particular unlabeled batch seen during training.

Semi-supervised describes the labeled/unlabeled mixture used to learn. Inductive and transductive describe the prediction goal; these terms are not exclusive names for learning algorithms. They also appear outside semi-supervised learning.

Detailed, reliable labels are often expensive to obtain. Weakly supervised learning uses the imperfect signals that are available instead, with methods designed for their limitations. Weak does not mean a weak model; it describes the supervision supplied to it.

Weak supervision describes learning with incomplete, inexact or inaccurate supervision. Incomplete supervision leaves some examples without labels. Inexact supervision provides a coarser answer than the task requires, such as saying a dog appears somewhere in an image without drawing its bounding box. Inaccurate supervision includes wrong labels, for example from an imperfect automatic rule.

These issues can overlap. A semi-supervised dataset has an incomplete set of labels; an automatically labeled dataset may have many labels but still contain mistakes. The next slide treats a different issue: a change in the data encountered at deployment.

Suppose labeled dog images from website A are well lit and sharp, while deployment images from website B are dark and compressed. The task can be unchanged even though the input distribution differs. This is domain shift; domain adaptation aims to adjust a model or representation to the target domain.

Possible steps include suitable normalization or learning a representation that aligns useful patterns across domains. Their success must be checked on representative target-domain examples. Having labeled A images and unlabeled B images can involve both semi-supervised learning and domain adaptation.

If website B also contains classes never represented in training, that is an additional problem: recognizing unfamiliar classes. A change in lighting and the appearance of new classes are distinct issues.

In reinforcement learning, an agent chooses actions in an environment. The state describes the current situation, a policy is the rule for choosing an action, and a reward is feedback about its consequence. An action can also change the state encountered next.

The objective concerns accumulated reward over time. In chess, an individual move may only be judged through a much later result. Training can use new interactions or previously collected experience; it does not require a correct action label for every situation.

The next example uses a simpler setting called a bandit: repeatedly choose one of several actions and observe a reward, without a changing game state. It isolates the exploration–exploitation decision.

Exploitation chooses the action currently estimated to have the best reward. Exploration tries an uncertain action to obtain information. Early estimates can be misleading when based on only a few noisy outcomes, so always exploiting can overlook a better option.

In the bandit example, ε (epsilon) is the probability of taking an exploratory random action. With probability 1 − ε, an ε-greedy policy takes an action with the highest current estimated reward. The reward estimate is updated from the observations received so far. Exploration can improve information, but a particular exploratory action need not give a good reward.

A multi-armed bandit is a repeated decision problem with several available actions and uncertain rewards. Each action is called an arm. Choosing an arm once is called a pull. These terms come from the lever, or arm, of a slot machine; the picture illustrates the naming.

In this lecture’s example, choose A, B or C. Observe a reward of 1 or 0 for the chosen action and update its estimated reward rate. The other actions’ outcomes are not revealed. Each arm has a fixed but initially unknown reward probability, so repeated choices of the same arm can produce different outcomes.

The learner wants a high total reward over many rounds. Exploitation uses the action currently estimated to be best; exploration tries an uncertain action to learn more. The upcoming question and interactive example use exactly this setup. Unlike the chess example, this bandit has no changing game state or delayed reward.

Each arm is an action with an unknown reward probability. Pulling an arm reveals a reward of 0 or 1. The displayed estimate summarizes the rewards observed for that arm so far; it is not its known true probability. In the plot, n is the number of times that arm has been pulled.

Try manual pulls, then an ε-greedy policy. After sampling each arm once, automated play chooses a random arm with probability ε and an arm with the highest current estimate otherwise. Compare ε = 0 with a positive value. Exploration can discover an underestimated action but does not guarantee a better reward on a particular pull.

The true rates can be revealed for teaching; the policy cannot use them. Seeds make the demonstration repeatable, Back replays recorded history, and Reset starts the chosen seed again. The limit is 60 pulls. This bandit has no changing game state, so it illustrates one decision tradeoff rather than the full complexity of reinforcement learning.

An application domain describes the kind of data and task. Computer vision works with images or video. Natural language processing works with text and language. Multimodal learning combines types of data, for example matching an image with its text caption, or using video together with audio.

A learning setting describes how feedback is obtained. A vision model could learn from human labels, unlabeled images or withheld image patches. Specifying “computer vision” therefore does not yet specify whether training is supervised, unsupervised or self-supervised.

Computer vision tasks can ask for different outputs from the same image. Classification predicts an image category. Object detection identifies objects and their locations, often with bounding boxes. Semantic segmentation assigns a class to each pixel.

The illustrated scene shows how the required output changes across these tasks. Its labels, boxes and regions are teaching illustrations, not predictions from a trained model. The output required by a task also affects the annotations needed for supervised training: an image category is less detailed than pixel-level labels.

Natural language processing includes tasks at several scales. Document topic classification might label a report as sports or politics. Part-of-speech tagging assigns grammatical roles, such as noun or verb, to tokens. Word-sense disambiguation identifies which meaning of a word is intended in context.

Compare “They bank the aircraft” with “They sat on the river bank”. The first uses bank as a verb; the second refers to a river’s edge. Context determines both grammatical use and meaning. A token is the unit of text processed by a model and may be a word or only part of one.

Multimodal learning combines different kinds of data. In an image–text task, one input may be a photograph and another a caption; the model can learn how the two relate, for example by learning to match an image with an appropriate caption. Audio together with video is another example.

The caption on the slide is a teaching description of the photograph. A multimodal application can use supervised pairs, targets derived from paired observations or multiple learning stages. Its data types and source of feedback should be described separately.