Representation: Learning What to Represent
Before a problem can be solved it must be described, and the description decides the difficulty. The right features make...
Month: July 2026
The Ergodic Hypothesis: Time Averages and Ensemble Averages
To predict a system's behavior one can watch it for a long time or imagine many copies at once. A...
Learning From Finite Data: The Problem of Induction
No finite set of examples logically compels a general conclusion. To infer a rule from cases is to leap beyond...
The Constructive Role of Noise
Noise is usually the enemy of signal, yet in learning and in nature it can help. A dose of randomness...
Optimization Landscapes: Hills, Valleys, and Saddles
To train a model is to descend a landscape of error, seeking its lowest point. The shape of that terrain,...
High-Dimensional Geometry: Where Intuition Breaks Down
Our intuitions are built for three dimensions and betray us in many. In high-dimensional spaces, volume concentrates strangely, distances flatten,...
Regression to the Mean
An extreme result tends to be followed by a less extreme one, not because of any force but because extremes...
Probability as Degree of Belief
What does it mean to say an event is probable? One view holds that probability is not a property of...
Infinity: Sizes Beyond Counting
Infinity is not a single vast number but a hierarchy of sizes, some larger than others. The counting numbers and...
Gödel: Truths a System Cannot Prove
Within any sufficiently rich system of mathematics there are true statements it cannot prove. Consistency and completeness cannot both be...
