A Tale of Perceptrons

Interesting history

In the 1950s, the nascent field of artificial intelligence was split into rival factions. Two groups competed for academic mindshare. The leading group was the “symbolists,” led by authorities such as Marvin Minsky and John McCarthy. The runners-up were the “connectionists,” led by Frank Rosenblatt.

The two groups had very different approaches. The symbolists believed in programming an intelligent machine from the ground up. Piece by piece, they planned to build computers that would eventually manipulate concepts faster and better than humans.

This idea might sound overly optimistic today, but it made perfect sense in the $1950s,$ when people were inventing the first high-level programming languages. Those languages seemed much closer to human thinking than plain old assembly language. Who knew how far that process could go? (In fact, John McCarthy invented one of the first high-level programming languages, LISP, in his quest to code intelligence.)

The opposite faction, the connectionists, chased another dream. Their idea could be summed up as: build a brain, and intelligence will come.

To simplify things, the brain is made of neurons connected through fibers. Each neuron has multiple input fibers and one output fiber. If the inputs are active in a certain pattern (maybe because they get a signal from the sensory organs), then the output is also activated. The connectionist leader Frank Rosenblatt built a machine inspired by that mechanism. BHe named the machine “perceptron” based on neurons and its final processing step “activation function.”

The first perceptron was very different from our tiny Python program. The Mark 1 perceptron was a room-sized piece of hardware that looked a bit like a server rack covered by an impenetrable tangle of wires. It had a camera connected to 400 photocells with very low-resolution pixels. The weights were implemented with potentiometers wired to the photocells. During the learning phase, the potentiometers were physically rotated by electric motors.

To overcome the perceptron’s limitations, the connectionists also studied multilayer perceptrons, which was able to tackle non-linearly separable data. Meanwhile, the symbolists were busy writing programs that solved algebra problems and stacked construction blocks with a robot arm.

To be fair, neither faction was making much progress toward intelligent machines. On the other hand, both factions were inclined to publicity and extravagant promises. At one point, Rosenblatt declared that the perceptron was the first step toward machines that would not only be smart, but even self-conscious. The popular press brought into it the public eye , which made symbolists jealous.

The feud went on for years, with the symbolists reaping the lion’s share of research funds, and the connectionists playing the part of the popular underdogs. Then, at some point, things really hit the fan.

The final battle