The Strange Math That Predicts (Almost) Anything

In 1905, socialist groups across Russia rose up against the Tsar, demanding political reform. This division extended into all sectors of society, including mathematics, where mathematicians picked opposing sides in the conflict.

• Pavel Nekrasov (pro-Tsar side): Deeply religious mathematician, unofficial Tsar of Probability, argued that math explained free will and God's will • Andrey Markov (socialist side): Atheist mathematician known as Andrey The Furious, criticized Nekrasov's work as mathematical abuse

For 200 years, probability theory relied on the law of large numbers, which only worked for independent events. Nekrasov claimed that observing this law in social statistics proved the underlying decisions were independent acts of free will.

Markov set out to prove that dependent events could also follow the law of large numbers, undermining Nekrasov's argument for free will and showing that probability could work with dependent events.

Markov analyzed the first 20,000 letters of Pushkin's Eugene Onegin, finding that 43% were vowels and 57% were consonants. He broke the text into overlapping pairs to examine letter dependencies.

• Vowel-vowel pairs appeared only 6% of the time, far less than the 18% predicted by independent probability • All letter pair combinations differed greatly from independent case predictions • This proved that letters were dependent on previous letters

Markov created a visual model with two states (vowel and consonant) and transition probabilities between them. A vowel had a 13% chance of transitioning to another vowel and 87% to a consonant.

When running the Markov chain prediction machine, the ratio of vowels to consonants initially jumped around but eventually converged to 43% vowels and 57% consonants, proving dependent systems could follow the law of large numbers.

In 1945, the United States detonated The Gadget, the world's first nuclear bomb, a 6-kilogram plutonium device. The Manhattan Project involved top scientists including Oppenheimer, von Neumann, and Stanislaw Ulam.

While recovering from severe encephalitis in 1946, Ulam played Solitaire and wondered about the probability of winning with 52 randomly-shuffled cards. The astronomical number of possible arrangements (52 factorial) made analytical solutions hopeless, so he proposed statistical approximation instead.

Scientists needed to understand how neutrons behave inside a nuclear core, where trillions of neutrons interact with their surroundings. Direct computation seemed impossible, but Ulam realized they could simulate these systems by generating random outcomes like Solitaire games.

• von Neumann recognized that unlike Solitaire, neutron behavior depends on previous conditions and position • They applied Markov chains to model neutron behavior, with states representing neutron conditions and transitions based on scatter, absorption, or fission events • They ran this on ENIAC, the world's first electronic computer, tracking the multiplication factor k to determine bomb feasibility • Named after the Monte Carlo Casino, the method became crucial for nuclear weapon development and reactor design

• In the mid-1990s, Yahoo became the most popular website by dominating search through massive marketing spending • Yahoo's keyword-based search was easily manipulated through keyword repetition and hidden white text • Search engines lacked quality ranking, only checking if pages mentioned search terms

Stanford PhD students Sergey Brin and Larry Page modeled the web as a Markov chain where webpages are states and links are transitions. Each link represented an endorsement, with value decreasing as pages sent out more links.

• A random web surfer starts on a page and follows links based on transition probabilities • Time spent on each page correlates with its importance ranking • Fake linking farms fail because pages with no external links don't distribute their votes • A 15% damping factor prevents surfers from getting stuck in disconnected loops by randomly jumping to pages

Launched in 1998 (initially called BackRub, then renamed Google after the number googol), the search engine overthrew Yahoo within four years. Alphabet, Google's parent company, is now worth around 2 trillion dollars, with a Markov chain at the heart of this trillion-dollar algorithm.

Claude Shannon, the father of information theory, extended Markov's text prediction work in the 1940s by using individual letters as predictors. He discovered that using the last two letters improved predictions, and using entire words as predictors produced sentences with short coherent sequences.

Shannon found that better predictions about the next word require considering more previous words. Sequences of four or more words often made sense even when the overall sentence was nonsensical.

• Gmail uses Markov chain-based algorithms to predict what users will type next • Large language models use tokens (letters, words, punctuation marks) as units in Markov-like structures • Unlike simple Markov chains, modern models use attention mechanisms to determine what previous context is relevant for predictions

• AI-generated text ending up on the internet as training data creates a feedback loop that can degrade language models • Positive feedback loops (like global warming's water vapor effect) make systems difficult to model with Markov chains • Systems with dependencies remain valuable for prediction where feedback loops aren't present

Despite extremely long histories of interactions and states, Markov chains show that you can ignore almost all past information and focus only on the current state. This memoryless property is what makes Markov chains so powerful for complex systems.

By making systems memoryless, Markov chains allow complex problems to be greatly simplified while still making meaningful predictions. Problem-solving often becomes a matter of constructing an appropriate Markov chain.

The core mathematical tool used in nuclear weapons, search engines, and language prediction emerged from a 100-year-old Russian political feud that had nothing to do with practical applications. Markov's determination to refute Nekrasov's philosophical argument about free will led directly to this foundational mathematical discovery.

• Card shuffling can be modeled as a Markov chain where deck arrangements are states and shuffles are steps • A riffle shuffle requires exactly 7 iterations to randomize a 52-card deck • Alternative shuffling methods require over 2,000 shuffles to achieve the same randomization