Future Computers Will Be Radically Different (Analog Computing)

For hundreds of years, analog computers were the most powerful computers on Earth, predicting eclipses, tides, and guiding anti-aircraft guns.

With the advent of solid-state transistors, digital computers took off and virtually every computer used today is digital.

• Incredibly powerful computing devices capable of completing many computations quickly • Require minimal power consumption compared to digital alternatives • Addition of two currents can be done by simply connecting two wires • Multiplication through passing current through a resistor (I × R)

• Not general-purpose computing devices; cannot run software like Microsoft Word • Cannot input exact values due to continuous inputs and outputs • Non-repeatable calculations never yield identical results • Manufacturing variations in components cause approximately 1% error • Single-purpose, inexact devices

The term artificial intelligence was coined in 1956; in 1958, Frank Rosenblatt built the perceptron to mimic how neurons fire in brains.

• Neurons can fire or not fire, represented as one or zero • Connections between neurons have variable strength and different weights • Connections are either excitatory (positive weights) or inhibitory (negative weights) • A neuron fires if the sum of activations times weights exceeds a bias threshold

The perceptron was trained by showing it circles and rectangles, adjusting weights accordingly. If output was correct, no change was made. If wrong, input activations were either added to or subtracted from the weights.

• Could distinguish between different shapes and letters • Media claimed it could walk, talk, see, write, and be conscious • In reality, the perceptron was limited and could not distinguish dogs from cats • Critics Minsky and Papert published a book in 1969 detailing limitations • Led to the first AI winter; Rosenblatt died in 1980

• In the 1980s, Carnegie Mellon created ALVINN, one of the first self-driving cars • ALVINN had a hidden layer of neurons between input and output • Input was 30 by 32-pixel images of the road ahead • Limited by computational speed; vehicle drove at 1-2 kilometers per hour

In the mid-2000s, researcher Fei-Fei Li proposed that neural networks needed more training data rather than better algorithms. From 2006 to 2009, she created ImageNet, a database of 1.2 million human-labeled images.

• From 2010 to 2017, ImageNet ran an annual competition for image classification • Images classified into 1,000 different categories including 90 dog breeds • In 2010, best performer had a top-5 error rate of 28.2% • By 2012, AlexNet from University of Toronto achieved 16.4% error rate

• Consisted of eight layers with 500,000 neurons total • Required adjusting 60 million weights and biases • Processing a single image required 700 million math operations • Pioneered GPU use for parallel computations • Now cited over 100,000 times; identified network scale as key to success

• Training neural networks requires electricity similar to yearly consumption of three households • Von Neumann Bottleneck: most time and energy spent fetching weight values rather than computing • Moore's Law limitations: transistor size approaching atom size causes fundamental physical challenges

• Neural networks are exploding in popularity and boil down to matrix multiplication • Neural networks do not need the precision of digital computers • Whether a network is 96% or 98% confident produces the same result • Slight variability in components and conditions can be tolerated

• Augmented and virtual reality for pose capture and rendering • Depth estimation from single webcam with heat mapping • Security cameras and autonomous systems • Manufacturing inspection equipment, such as detecting defective Fritos • Smart home speakers to listen for wake words with minimal power

• Mythic AI uses repurposed digital flash storage cells as variable resistors • Floating gate electrons are partially stored to adjust resistance of the channel • Current flow equals V/R, which equals voltage times conductance • Single flash cell multiplies two values: voltage times conductance • Cells wired together so currents add, completing matrix multiplication

Mythic's first product can perform 25 trillion math operations per second while burning about three watts of power.

Newer digital systems can perform 25 to 100 trillion operations per second, but are large thousand-dollar systems consuming 50 to 100 watts of power.

• Training algorithms still requires large GPU hardware • Analog chips ideal for deploying already-trained AI workloads • Suitable for security cameras, autonomous systems, and inspection equipment • Could replace digital circuitry in smart home speakers for wake-word detection

When performing 50 sequences of matrix multiplications entirely in analog domain, signal becomes distorted. Solution involves converting between analog and digital domains after each processing block to preserve signal integrity.

Rosenblatt found his digital IBM computer too slow and built a custom analog computer with variable resistors and motors to drive them. His neural network ideas were correct, and perhaps he was right about analog as well.

While uncertain whether analog computers will take off like digital did in the last century, they do seem better suited to many tasks computers need to perform today.

• Everything from music to pictures to video went digital in the last 50 years • Digital was always thought to be the optimal way to process information • In 100 years, digital may be seen as a starting point, not an endpoint • Human brains are digital (neurons fire or don't) and analog (thinking happens everywhere at once)

To achieve true artificial intelligence with machines that think like humans, we may need the power of analog computing rather than relying solely on digital systems.