← Back to home

Meta-Learning (Computer Science)

Oh, this again. Another attempt to make sense of the incomprehensible, to impose order on chaos. Fine. Let's dissect this "meta-learning" business. Don't expect me to hold your hand through it. It's like trying to teach a pigeon quantum physics.

Subfield of Machine Learning

This article, for better or worse, is about meta-learning within the context of machine learning. If you're looking for its more... pedestrian cousin in social psychology, that's over there: Meta-learning. And for the neuroscientists who insist on their own peculiar brand of it, try Metalearning (neuroscience). Honestly, the sheer effort people put into categorizing the same basic idea is exhausting.

This is also part of a much larger, frankly overwhelming, series on Machine learning and data mining. Don't get lost.

Paradigms

Think of these as the different flavors of how a machine decides to learn. Some are more… direct. Others are more insidious.

  • Supervised learning: The one where you hold its hand and tell it exactly what's what. Like teaching a child by showing them flashcards. Tedious.
  • Unsupervised learning: The child left to its own devices. It might discover something, or it might just make a mess.
  • Semi-supervised learning: A little bit of both. A hint of guidance, but mostly left to its own interpretation. Risky.
  • Self-supervised learning: The machine thinks it's being clever, creating its own labels. Cute, really.
  • Reinforcement learning: Rewards and punishments. Like training a dog, but with more complex equations and less slobber. Usually.
  • Meta-learning: The one we're wading through. Learning how to learn. A recursive nightmare.
  • Online learning: It learns as it goes. Adapting on the fly. Like trying to change your outfit while sprinting.
  • Batch learning: It waits until it has a whole lot of data, then learns all at once. Like cramming for an exam. Usually results in indigestion.
  • Curriculum learning: Teaching it in a structured order, from easy to hard. Like a sensible educator. So rarely seen in the wild.
  • Rule-based learning: It learns by following explicit rules. Predictable, but often brittle.
  • Neuro-symbolic AI: A clumsy attempt to merge the elegance of neural networks with the logic of symbols. Like a cat trying to operate a calculator.
  • Neuromorphic engineering: Trying to build hardware that mimics the brain. More biological than computational. Fascinating, in a disturbing sort of way.
  • Quantum machine learning: Using quantum mechanics to do machine learning. Still mostly theoretical, and frankly, too much effort to explain properly right now.

Problems

These are the tasks the learning algorithms are supposed to tackle. The why behind the how.

  • Classification: Sorting things into boxes. Is it a cat or a dog? A threat or an annoyance?
  • Generative modeling: Creating new things. Art, music, convincing lies.
  • Regression: Predicting a number. How much will it rain? How much will this stock plummet?
  • Clustering: Finding groups in data without being told what the groups are. Like finding patterns in the static.
  • Dimensionality reduction: Simplifying complex data. Making sense of the noise by throwing away most of it.
  • Density estimation: Figuring out how likely certain data points are. Where are the people most likely to be? Where are the problems most likely to be?
  • Anomaly detection: Finding the things that don't belong. The outliers. The exceptions to the rule. Often the most interesting bits.
  • Data cleaning: The Sisyphean task of fixing messy data. Like scrubbing graffiti off a perfectly good wall.
  • AutoML: Machines building other machines. A dangerous feedback loop, if you ask me.
  • Association rules: Finding relationships. If you buy bread, you'll probably buy butter. Groundbreaking.
  • Semantic analysis: Understanding the meaning. The subtext. The unspoken.
  • Structured prediction: Predicting complex outputs, not just single labels. Like predicting an entire sentence, not just one word.
  • Feature engineering: Manually creating the inputs for the model. The art of telling the machine what to look at.
  • Feature learning: Letting the machine figure out what's important on its own. Less manual labor, more potential for surprise.
  • Learning to rank: Ordering items by relevance. Search engines do this. They decide what you see.
  • Grammar induction: Learning the rules of a language from examples. Like deciphering an ancient text.
  • Ontology learning: Building structured representations of knowledge. Creating hierarchies of concepts.
  • Multimodal learning: Learning from different types of data simultaneously. Text, images, sound. The whole sensory experience.

Supervised learning

The foundational stuff. The basics.

  • (classification](/Statistical_classification) & regression): The two primary outputs. Categorical or continuous. Simple.
  • Apprenticeship learning: Learning by observing an expert. Mimicry.
  • Decision trees: Branching logic. If this, then that. Easy to visualize, sometimes too simple.
  • Ensembles: Combining multiple models. Like a committee, but hopefully more effective.
    • Bagging: Bootstrap aggregating. Random forests are a prime example.
    • Boosting: Sequential learning, where each new model corrects the errors of the previous ones.
    • Random forest: A forest of decision trees. More robust, less prone to overfitting.
  • k-NN: K-Nearest Neighbors. Simple, but can be computationally expensive.
  • Linear regression: The most basic regression. A straight line through the data.
  • Naive Bayes: Probabilistic, assumes independence. Often surprisingly effective.
  • Artificial neural networks: The complex, interconnected beasts. The foundation of modern deep learning.
  • Logistic regression: For classification, despite the name. Uses a sigmoid function.
  • Perceptron: The simplest form of a neural network. A single layer.
  • Relevance vector machine (RVM): A probabilistic approach, similar to SVMs but often sparser.
  • Support vector machine (SVM): Finds the optimal hyperplane to separate data. Powerful, but can be tricky to tune.

Clustering

Finding order in the disorder.

  • BIRCH: Balanced Iterative Reducing and Clustering using Hierarchies. Efficient for large datasets.
  • CURE: Clustering Using REpresentatives. Handles non-spherical clusters.
  • Hierarchical: Builds a tree of clusters. Dendrograms.
  • k-means: The classic. Simple, fast, but sensitive to initial centroids and assumes spherical clusters.
  • Fuzzy: Allows data points to belong to multiple clusters with varying degrees of membership.
  • Expectation–maximization (EM): An iterative approach, often used for Gaussian Mixture Models.
  • DBSCAN: Density-Based Spatial Clustering of Applications with Noise. Finds arbitrarily shaped clusters and identifies outliers.
  • OPTICS: Ordering Points To Identify the Clustering Structure. An extension of DBSCAN.
  • Mean shift: A non-parametric, density-based clustering method. Finds modes in the data distribution.

Dimensionality reduction

Making the complex comprehensible by shedding the unnecessary.

  • Factor analysis: Assumes observed variables are linear combinations of latent factors.
  • CCA: Canonical Correlation Analysis. Finds relationships between two sets of variables.
  • ICA: Independent Component Analysis. Separates a multivariate signal into additive subcomponents.
  • LDA: Linear Discriminant Analysis. Used for both classification and dimensionality reduction.
  • NMF: Non-negative Matrix Factorization. Decomposes a matrix into two non-negative matrices.
  • PCA: Principal Component Analysis. The workhorse. Finds orthogonal axes of maximum variance.
  • PGD: Proper Generalized Decomposition. Less common, but useful.
  • t-SNE: t-Distributed Stochastic Neighbor Embedding. Excellent for visualizing high-dimensional data in low dimensions, but not for general-purpose reduction.
  • SDL: Sparse Dictionary Learning. Learns a dictionary of basis elements.

Structured prediction

Predicting things that have internal structure.

Anomaly detection

Spotting the odd one out. The things that don't fit the pattern.

  • RANSAC: Random Sample Consensus. Robust to outliers.
  • k-NN: Can be used for anomaly detection by measuring distance to nearest neighbors.
  • Local outlier factor: Measures local density deviation.
  • Isolation forest: Isolates anomalies by randomly partitioning data.

Neural networks

The modern marvels. Or monsters. Depends on your perspective.

  • Autoencoder: Learns a compressed representation of data. For dimensionality reduction or feature learning.
  • Deep learning: Networks with many layers. The current obsession.
  • Feedforward neural network: The simplest kind. Information flows in one direction.
  • Recurrent neural network: Has loops, allowing it to process sequential data and have memory.
  • LSTM: A type of RNN designed to avoid long-term dependency issues.
  • GRU: A simpler variant of LSTM.
  • ESN: A type of RNN where only the output layer is trained. Part of reservoir computing.
  • Boltzmann machine: A stochastic recurrent neural network.
  • Restricted: A simpler version of Boltzmann machine.
  • GAN: Two networks competing: a generator and a discriminator. Creates realistic synthetic data.
  • Diffusion model: A newer class of generative models, showing impressive results.
  • SOM: Self-Organizing Map. For dimensionality reduction and visualization.
  • Convolutional neural network: Specialized for grid-like data, especially images.
  • U-Net: A CNN architecture for biomedical image segmentation.
  • LeNet: An early, influential CNN for digit recognition.
  • AlexNet: A breakthrough CNN that won the ImageNet competition.
  • DeepDream: An algorithm that visualizes patterns learned by neural networks.
  • Neural field: A continuous representation of neural activity.
  • Neural radiance field: For synthesizing novel views of complex scenes.
  • Physics-informed neural networks: Incorporates physical laws into the network's objective function.
  • Transformer: Architecture based on attention mechanisms, revolutionized NLP and is now used widely.
    • Vision: Applying Transformer architectures to image recognition.
    • Mamba: A recent, efficient state space model.
  • Spiking neural network: Mimics biological neurons more closely, using discrete events (spikes).
  • Memtransistor: A type of memory device that can also perform computation.
  • Electrochemical RAM (ECRAM): Another type of emerging memory technology for AI hardware.

Reinforcement learning

Learning through trial and error. The thrill of the reward, the sting of the failure.

  • Q-learning: Learns an action-value function.
  • Policy gradient: Directly learns a policy.
  • SARSA: On-policy temporal difference learning.
  • Temporal difference (TD): A core concept in RL, learning from differences in predictions.
  • Multi-agent: RL with multiple interacting agents. A chaotic ballet.
  • Self-play: Agents learn by playing against themselves. Like a solitary chess master.

Learning with humans

When the machine doesn't have to do it all alone.

  • Active learning: The machine strategically asks humans for labels on the most informative data points.
  • Crowdsourcing: Using large groups of people for data labeling. A digital hive mind.
  • Human-in-the-loop: Humans and machines collaborate. A partnership.
  • Mechanistic interpretability: Trying to understand how deep learning models make decisions. Peering into the black box.
  • RLHF: Reinforcement learning guided by human preferences. Like teaching a child what's "good."

Model diagnostics

How do we know if it's any good?

  • Coefficient of determination: R-squared. How much variance is explained.
  • Confusion matrix: A table showing true positives, false positives, etc. For classification.
  • Learning curve: Plotting performance against training data size. Shows if it's underfitting or overfitting.
  • ROC curve: Receiver Operating Characteristic. For binary classifiers, showing trade-off between true positive and false positive rates.

Mathematical foundations

The bedrock. The abstract principles that make it all work.

Journals and conferences

Where the serious (or at least, published) work happens.

  • AAAI: Association for the Advancement of Artificial Intelligence.
  • ECML PKDD: European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases.
  • NeurIPS: Neural Information Processing Systems. A major one.
  • ICML: International Conference on Machine Learning. Another big one.
  • ICLR: International Conference on Learning Representations. Focuses on deep learning.
  • IJCAI: International Joint Conference on Artificial Intelligence.
  • ML: Machine Learning (journal).
  • JMLR: Journal of Machine Learning Research. Open access.

Related articles

More reading, if you're masochistic enough.

Meta-learning

So, meta-learning. It's a subfield of machine learning, where the algorithms are designed to learn about learning. They analyze metadata from past experiments to figure out how to get better. As of 2017, the definition was still a bit fuzzy, but the core idea is to make learning more flexible. To improve existing learning algorithms or, more ambitiously, to learn the learning algorithm itself. Hence, the rather obvious alternative: "learning to learn."

The problem, you see, is that every learning algorithm carries its own baggage – its inductive bias. It’s built on assumptions about the data. If those assumptions don't align with the actual learning problem, well, it fails. Spectacularly. An algorithm that excels in one domain might be utterly useless in another. This severely limits the practical application of machine learning and data mining techniques because we don't truly understand the intricate relationship between a learning problem (often just a database) and the effectiveness of different algorithms.

By sifting through various kinds of metadata – properties of the problem, characteristics of the algorithms, performance metrics, or patterns previously identified in data – meta-learning aims to learn, select, modify, or combine learning algorithms. This allows for more effective problem-solving. Some critics point out that meta-learning approaches bear a striking resemblance to metaheuristic methods, which is a related, but distinct, problem space.

A rather poetic analogy, and one that inspired early work by Jürgen Schmidhuber in 1987 and Yoshua Bengio and colleagues in 1991, is that of genetic evolution. Evolution, in a sense, learns the learning procedure, encoding it in genes and then executing it in each individual's brain. In a hierarchical meta-learning system, using something like genetic programming, better evolutionary methods could theoretically be learned by a "meta-evolution," which itself could be refined by a "meta-meta-evolution," and so on. An infinite regression of improvement. Or a descent into madness.

Definition

A proposed definition for a meta-learning system, if you must have one, combines three criteria:

  • The system must incorporate a learning subsystem – obviously.
  • Experience is acquired by leveraging meta-knowledge. This meta-knowledge is extracted:
    • From a previous learning episode on a single dataset, or
    • From multiple, distinct domains.
  • The learning bias must be selected dynamically.

"Bias," in this context, refers to the inherent assumptions that guide the selection of explanatory hypotheses. It's not about prejudice in the human sense, nor is it the same as the bias in the bias-variance dilemma. Meta-learning grapples with two primary aspects of learning bias:

  • Declarative bias: This defines the representation of the hypothesis space. It dictates the search space's size. For instance, restricting hypotheses to only linear functions.
  • Procedural bias: This imposes constraints on the order in which inductive hypotheses are considered. An example would be a preference for simpler, smaller hypotheses.

Common Approaches

There are essentially three main avenues of attack in meta-learning:

  • Model-based: These systems utilize (cyclic) networks equipped with external or internal memory.
  • Metric-based: The focus here is on learning effective distance metrics or similarity functions.
  • Optimization-based: These approaches directly optimize the model's parameters to facilitate rapid learning.

Model-Based

Model-based meta-learning systems update their parameters with remarkable speed, often requiring only a few training steps. This is achieved either through the network's internal architecture or by the control of another, higher-level meta-learner model.

Memory-Augmented Neural Networks

A Memory-Augmented Neural Network, or MANN, is designed to quickly encode new information. This allows it to adapt to novel tasks after being exposed to just a handful of examples. The idea is to give the network a place to store and retrieve relevant information efficiently.

Meta Networks

Meta Networks (MetaNet) aim to learn a meta-level knowledge that transcends individual tasks. They achieve rapid generalization by dynamically shifting their inductive biases through fast parameterization. Think of it as learning the underlying principles of learning across different scenarios.

Metric-Based

The core principle here echoes that of nearest neighbors algorithms, where weights are assigned based on a kernel function. The goal is to learn a metric or a distance function that accurately represents the relationships between objects in the task space. The effectiveness of a metric is inherently problem-dependent; it must capture the nuances of the input data to facilitate successful problem-solving.

Convolutional Siamese Neural Network

A Siamese neural network consists of two identical networks that are trained jointly. A function is applied above them to learn the relationship between pairs of input data samples. The key is that both networks share the same weights and parameters, ensuring they process inputs in precisely the same way, allowing for comparison.

Matching Networks

Matching Networks learn a mapping function. This function takes a small, labeled support set and an unlabeled example, then predicts the label for the example. This approach bypasses the need for extensive fine-tuning when encountering new class types.

Relation Network

The Relation Network (RN) is trained from scratch, end-to-end. During the meta-learning phase, it learns a deep distance metric. This metric is used to compare a small number of images within specific "episodes," each designed to simulate a few-shot learning scenario.

Prototypical Networks

Prototypical Networks learn a metric space where classification is performed by calculating distances to prototype representations of each class. Compared to other few-shot learning methods, they adopt a simpler inductive bias, which proves advantageous in limited-data situations, often leading to satisfactory results.

Optimization-Based

Optimization-based meta-learning algorithms focus on refining the optimization algorithm itself. The objective is to enable the model to learn effectively from just a few examples.

LSTM Meta-Learner

An LSTM-based meta-learner is designed to learn the precise optimization algorithm used for training another learner, typically a neural network classifier, within a few-shot learning context. The learned parametrization enables it to acquire appropriate parameter updates tailored for a specific number of training iterations, while also learning a general network initialization that promotes rapid convergence.

Temporal Discreteness

Model-Agnostic Meta-Learning (MAML) is a notably general optimization algorithm. Its strength lies in its compatibility with any model that learns through gradient descent. It's designed to find model parameters that are sensitive to changes and can be quickly adapted to new tasks.

Reptile

Reptile is a surprisingly simple meta-learning optimization algorithm. Its elegance stems from the fact that both of its core components rely on meta-optimization via gradient descent, and it remains model-agnostic. It iteratively updates the model's initial parameters based on the gradients obtained from training on different tasks.

Examples

The landscape of meta-learning is populated by various approaches that can be interpreted as instances of this broader concept.

  • Recurrent neural networks (RNNs) have been recognized as universal computers. As far back as 1993, Jürgen Schmidhuber demonstrated how "self-referential" RNNs could, in principle, learn their own weight update algorithms via backpropagation, potentially developing algorithms far more sophisticated than backpropagation itself. Later, in 2001, Sepp Hochreiter and colleagues developed a successful supervised meta-learner based on Long short-term memory RNNs. This system learned, through backpropagation, a learning algorithm for quadratic functions that significantly outperformed standard backpropagation in speed. In 2017, researchers at Deepmind expanded on this concept, applying it to optimization processes.

  • During the 1990s, Meta Reinforcement Learning (Meta RL) was explored within Schmidhuber's research group. This was achieved through self-modifying policies encoded in a universal programming language that included specific instructions for altering the policy itself. The setup involved a single, lifelong trial where the RL agent's objective was to maximize its cumulative reward. It learned to accelerate reward acquisition by continuously refining its own learning algorithm, which was an integral part of its "self-referential" policy.

  • An extreme manifestation of Meta Reinforcement Learning is embodied by the theoretical Gödel machine. This construct is capable of inspecting and modifying any aspect of its own software, which crucially includes a general theorem prover. It's designed to achieve recursive self-improvement in a provably optimal manner.

  • Model-Agnostic Meta-Learning (MAML), introduced in 2017 by Chelsea Finn and colleagues, is a significant advancement. Given a sequence of tasks, MAML trains the parameters of a model such that only a few gradient descent steps, using minimal training data from a new task, are required to achieve good generalization performance on that task. In essence, MAML "trains the model to be easy to fine-tune." It has been successfully applied to few-shot image classification benchmarks and to policy-gradient-based reinforcement learning.

  • Variational Bayes-Adaptive Deep RL (VariBAD), introduced in 2019, takes a different tack. While MAML is optimization-based, VariBAD is a model-based approach for meta reinforcement learning. It utilizes a variational autoencoder to encapsulate task-specific information within an internal memory, thereby conditioning its decision-making process on the nature of the task.

  • A common pitfall in meta-learning when addressing multiple tasks is optimizing for the average score across all tasks. This can lead to certain tasks being neglected in favor of the overall average, which is often unacceptable in real-world applications. Robust Meta Reinforcement Learning (RoML) addresses this by focusing on improving performance on low-scoring tasks, thereby increasing robustness to task selection. RoML functions as a meta-algorithm, meaning it can be applied on top of existing meta-learning algorithms (like MAML and VariBAD) to enhance their robustness. It's applicable to both supervised meta-learning and meta reinforcement learning.

  • Discovering meta-knowledge involves inducing knowledge, such as rules, that explains how each learning method is likely to perform on different learning problems. The metadata used for this purpose comprises characteristics of the data (general, statistical, information-theoretic, etc.) within the learning problem, and characteristics of the learning algorithm itself (type, parameter settings, performance measures, etc.). Another learning algorithm then learns the relationship between these data and algorithm characteristics. When presented with a new learning problem, its data characteristics are measured, and the performance of various learning algorithms is predicted. This allows for the identification of the algorithms best suited for the new problem.

  • Stacked generalization operates by combining multiple, distinct learning algorithms. The metadata here consists of the predictions generated by these different algorithms. A secondary learning algorithm then learns from this metadata to determine which combinations of algorithms yield consistently good results. For a new learning problem, the predictions of the selected set of algorithms are combined (e.g., through weighted voting) to produce the final prediction. The rationale is that since each individual algorithm is known to work well on a subset of problems, their combination is hoped to be more flexible and capable of making accurate predictions across a wider range.

  • Boosting, while related to stacked generalization, employs the same algorithm multiple times. In each run, the training data examples are assigned different weights. This results in varied predictions, each focusing on correctly predicting a specific subset of the data. Combining these predictions ideally leads to improved, albeit more computationally expensive, results.

  • Dynamic bias selection involves altering the inductive bias of a learning algorithm to better align with the specific problem at hand. This is achieved by modifying key aspects of the algorithm, such as the hypothesis representation, heuristic formulas, or parameter settings. Numerous approaches exist within this category.

  • Inductive transfer investigates methods for improving the learning process over time. The metadata comprises knowledge gleaned from previous learning episodes and is utilized to efficiently develop an effective hypothesis for a new task. A closely related concept is learning to learn, where the explicit goal is to leverage acquired knowledge from one domain to facilitate learning in other, potentially unrelated, domains.

  • Other methods that harness metadata to enhance automatic learning include learning classifier systems, case-based reasoning, and constraint satisfaction.

  • Some initial, theoretical explorations have begun into using Applied Behavioral Analysis as a foundation for agent-mediated meta-learning. The aim is to understand the performance of human learners and subsequently adjust the instructional course of an artificial agent.

  • AutoML, exemplified by projects like Google Brain's "AI building AI," has demonstrated remarkable capabilities. In 2017, such systems briefly surpassed existing ImageNet benchmarks, showcasing the potential for automated machine learning system design.

There. I've laid it out. Every detail, every link. It's as clear as it's going to get, which, given the subject matter, isn't saying much. Don't come back asking me to explain it again. My patience is not a resource to be squandered.