Catastrophic Forgetting: The Price of Learning Anew
To learn a new thing can be to lose an old one. A system trained on a fresh task may...
Year: 2026
Disentanglement: Factoring the World Into Its Causes
The world's appearances arise from a few underlying factors, mixed together. A face varies by identity, lighting, and pose; a...
Representation: Learning What to Represent
Before a problem can be solved it must be described, and the description decides the difficulty. The right features make...
The Ergodic Hypothesis: Time Averages and Ensemble Averages
To predict a system's behavior one can watch it for a long time or imagine many copies at once. A...
Learning From Finite Data: The Problem of Induction
No finite set of examples logically compels a general conclusion. To infer a rule from cases is to leap beyond...
The Constructive Role of Noise
Noise is usually the enemy of signal, yet in learning and in nature it can help. A dose of randomness...
Optimization Landscapes: Hills, Valleys, and Saddles
To train a model is to descend a landscape of error, seeking its lowest point. The shape of that terrain,...
High-Dimensional Geometry: Where Intuition Breaks Down
Our intuitions are built for three dimensions and betray us in many. In high-dimensional spaces, volume concentrates strangely, distances flatten,...
Regression to the Mean
An extreme result tends to be followed by a less extreme one, not because of any force but because extremes...
Probability as Degree of Belief
What does it mean to say an event is probable? One view holds that probability is not a property of...
