The History of AI and Machine Learning

1.1  The founding conjecture

Most scientific fields have to be reconstructed backwards by historians, their beginnings argued over and never settled. Artificial intelligence is unusual: it has a birth certificate, and the certificate is a grant application.

In the late summer of 1955, four researchers — John McCarthy of Dartmouth College, Marvin Minsky, Nathaniel Rochester of IBM, and Claude Shannon of Bell Laboratories — submitted A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence to the Rockefeller Foundation.1 The document did two things that still shape the industry. It coined a name for the field. And it stated a conjecture that the entire enterprise has been operating on ever since:

“…every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.”1

Read that sentence slowly, because it is doing more work than it appears to. It is not a claim that machines are intelligent, nor a prediction about when they will be. It is a claim about describability — that intelligence contains no ingredient which resists precise specification. Everything that has happened since, including everything in the remaining fourteen lessons of this course, is a series of attempts to cash that cheque. It has never been proven and it has never been refuted.

The material details are worth having, because they puncture the myth of the heroic well-funded moonshot. The four proposers asked the Rockefeller Foundation for $13,500. They received $7,500 — a little over half.2 The proposal envisaged “a 2 month, 10 man study.” What actually happened in Hanover, New Hampshire in the summer of 1956 was looser than that: participants came and went, and the workshop is variously described in the historical record as running five, six, or eight weeks.3

FIG-01-01

The founding document

A stylised period illustration of the 1955 proposal document lying on a desk, with the conjecture sentence rendered legibly in mid-century typewriter face and set apart by a hand-drawn box. Around it, faint pencil marginalia in four different hands, suggesting four authors. Deliberately understated — this should feel like an archival photograph, not a triumphal poster. No robots, no glowing brains, no circuit-board motifs anywhere in this course's illustration set.

Aspect 3:2 · 2400×1600 px · Brand palette: navy #00040c, silver-blue #a8c0d8, column gold #a87848, paper #f8f7f4 · Style: archival editorial illustration

Two earlier landmarks made the proposal thinkable. In 1943, Warren McCulloch and Walter Pitts published a mathematical model of the neuron, showing that networks of simple threshold units could compute logical functions — the first formal bridge between brain and computation.4 In 1950, Alan Turing sidestepped the unanswerable question “can machines think?” by replacing it with an operational one: could a machine sustain a conversation well enough that an interrogator could not reliably tell it from a human?5

1.2  Two programmes, one field

It is tempting to narrate AI's history as a single lineage that got progressively better. That narrative is wrong, and getting it wrong makes the present incomprehensible. From almost the beginning the field carried two rival research programmes, and the current moment is best understood as the decisive victory of the one that kept losing.

The symbolic programme

The first bet was that intelligence is symbol manipulation. Encode enough facts, rules, and logical machinery, and reasoning follows. The programme had early and genuine wins. Allen Newell, Herbert Simon, and Cliff Shaw's Logic Theorist, demonstrated at Dartmouth in 1956, proved 38 of the first 52 theorems in Whitehead and Russell's Principia Mathematica — and found, for one of them, a proof shorter than the original.6 McCarthy designed Lisp in 1958, giving the field a language in which programs and data had the same shape.7 Joseph Weizenbaum's ELIZA in 1966 simulated a Rogerian psychotherapist using little more than pattern substitution — and disturbed its own author by how readily people confided in it, a result this course returns to in Lesson 4.8

The connectionist programme

The rival bet was that intelligence emerges from many simple units adjusting their connections. Frank Rosenblatt's perceptron, introduced in 1958, was a learning machine: it adjusted numerical weights in response to examples rather than executing rules a human had written.9 In 1969, Minsky and Seymour Papert published Perceptrons, proving that a single-layer perceptron could not represent even the exclusive-or function.10 The result was mathematically correct and narrow in scope — it said nothing about multi-layer networks — but its reception was neither. Funding and attention drained away from connectionist work for roughly fifteen years.

The unlock, when it came, was a method for assigning credit across the layers of a deeper network: backpropagation, popularised by David Rumelhart, Geoffrey Hinton, and Ronald Williams in Nature in 1986.11 This is the algorithm that trains essentially every model discussed in this course. It arrived thirty-six years before ChatGPT.

FIG-01-02

Two braids, not one thread

A diagram of two parallel strands running left to right across seventy years, one labelled Symbolic and one labelled Connectionist. Each strand thickens and thins to represent funding and attention. Key events sit as nodes on their own strand: Logic Theorist 1956, Lisp 1958, ELIZA 1966, expert systems 1980s on the symbolic strand; McCulloch–Pitts 1943, Perceptron 1958, Perceptrons 1969 (strand pinches almost to nothing), Backpropagation 1986, AlexNet 2012, Transformer 2017 on the connectionist strand. The connectionist strand should be visibly thinner than the symbolic one until roughly 2012, then dominant. The point of the image is the crossover, so make that moment unmistakable.

Aspect 16:9 · 3200×1800 px · Line diagram, two-colour plus accent · Must remain legible at 90 mm wide in print

1.3  The winters

An AI winter is a period in which funding, hiring, and institutional confidence collapse after a stretch of over-promising. There have been two, and the mechanism was the same both times: demonstrations that worked beautifully on small, tidy problems failed to scale to real ones, and the gap between what had been promised and what was delivered became impossible to fund past.

The first winter's proximate cause is unusually well documented. In 1973 the applied mathematician Sir James Lighthill delivered Artificial Intelligence: A General Survey to the British Science Research Council. Lighthill's central charge was combinatorial explosion: techniques that succeeded on toy problems ran into possibility-spaces that grew so fast with problem size that no feasible amount of computer time could search them. He recommended against funding the central ambition of general machine intelligence. Within months the SRC withdrew support from AI work at most British universities, and a research community that had been world-class was effectively dismantled.12

The second winter arrived in the late 1980s. Commercial expert systems — programs encoding a specialist's rules for a narrow domain — had been the boom of that decade. They worked, within limits, and then ran into limits that were structural rather than technical: the rule bases were brittle at the edges of their domain, and they were expensive to maintain, because every change in the world required a human to hand-edit logic. The specialised Lisp hardware market collapsed in 1987, and funding contracted again.13

1.4  The statistical turn

Here is the conceptual move the rest of the course rests on. Say it slowly, and check that you can restate it in your own words before continuing.

This is the shift from programming to training. It sounds like a technicality and it is a change of kind. Under the classical model, a system's competence is bounded by what its authors could articulate — and it turns out that most of what humans know how to do, we cannot articulate. You recognise a friend's face in a crowd; try writing down the rule. This is Polanyi's paradox: we know more than we can tell. The symbolic programme kept colliding with it, because a rule-writing approach can only capture the knowledge someone can put into words.

Learning from examples routes around the paradox. You no longer need to state the rule; you need to supply cases where the answer is known. That is why the tasks AI conquered in the order it did — perception first, then language, then multi-step reasoning — track how easy it is to collect labelled examples, not how easy the task feels to a human.

Three ingredients had to arrive together for this to work at scale, and only one of them was an idea:

1. Method — backpropagation and the layered architectures it makes trainable, in place since 1986.11
2. Data — labelled corpora large enough to constrain millions of parameters, which the consumer internet supplied after roughly 2000.
3. Compute — graphics processors, built for rendering, turning out to be well suited to the dense linear algebra that training requires.

All three converged publicly in 2012, when a deep convolutional network won the ImageNet competition with a top-5 error rate of 15.3% against 26.2% for the next-best entry — a margin that ended the argument about whether learned features beat hand-designed ones.14 Lesson 2 treats that result and its successors as evidence rather than headlines.

Demonstration protocol

For the recorded lesson, and reproducible by the learner. Record the failure if there is one.

1. Set up the contrast. Ask a current model a question whose answer is strictly deducible — a small logic or arithmetic puzzle it cannot have memorised. Then ask one that is purely statistical — what word most naturally completes a common phrase.
2. Show both succeeding. Do not editorialise yet.
3. Make the point. The machinery underneath was identical in both cases. The model did not switch into a “logic mode.” It predicted, in both. This plants the seed that Lesson 3 harvests.
4. Push until it breaks. Extend the deducible problem — more steps, larger numbers — until the model fails. Leave the failure in. Note where the boundary was.

Why step 4 is not optional. A demonstration that only shows successes teaches the learner to trust the system, which is the opposite of the calibration this course is trying to build. The boundary is the lesson.