8 Learning to Program

Programs as Recipes

Programming is frequently described as telling the computer what to do. Presupposed are specific roles for the programmer and the computer. The programmer can, for instance, be understood as a chef who is in charge of a kitchen and the computer is anthropomorphized as a staff member of the kitchen, a cook or sous-chef who follows the directions of the chef. Programming languages, then, can be described as the set of terms that are “understood” by the computer, including the means the programmer uses to tell the computer what to do. Finally, programs (implementations of algorithms) are step-by-step instructions written in a programming language, a circumscribed form of language with verbs that can only be conjugated in the imperative (do this) or in the conditional (if this, then do that). In this compound metaphor, programs and algorithms are troped as “recipes”; they are the directions the chef gives to the staff (see Sack 2019, 84).

If the computer is seen to be solely a device of automation, then the only learning accomplished in this scenario is done by the computer or staff member who takes instructions. Alternatively, if the computer is presented as an instrument of education, it is the programmer—the chef or writer of the “recipes”—who learns. In his 1971 lectures at the American Museum of Natural History in New York City, John Kemeny, a past president of Dartmouth College and co-designer of the BASIC programming language, advocated for the creation of programming languages that allow the student to learn by “teaching” (i.e., programming) the computer: “At Dartmouth we have seen hundreds of examples of spectacular success of learning through teaching the computer” (Kemeny 1972, 79).

Contemporary, algorithmic, AI-inflected conditions resurrect a question as old as the earliest forms of automation: If a machine can do it, why should students learn how to do it? This memo argues that students need to learn how to do it—arithmetic, essay composition, coding, drawing, etc.—even if they will not spend most of their time doing it. If, conversely, know-how becomes exclusively the province of machines—AI algorithms or others—we risk becoming a world of know-nothings without the critical faculties to judge whether what is done is being done well. In this sense, artificial intelligence (AI) poses not an existential but an epistemic risk.

Programming as a Medium of Expression

From Kemeny’s perspective, a programming language like BASIC is not, primarily, a machine of postindustrial automation but a language or medium of expression. In an essay, “Why Programming is a Good Medium for Expressing Poorly Understood and Sloppily Formulated Ideas,” published five years before Kemeny’s lecture, the computer scientist and cofounder of AI Marvin Minsky put it like this: “A computer is like a violin. You can imagine a novice trying first a phonograph and then a violin. The latter, he says, sounds terrible. That is the argument we have heard from our humanists and most of our computer scientists. Computer programs are good, they say, for particular purposes, but they aren’t flexible. Neither is a violin, or a typewriter, until you learn how to use it” (Minsky 1967). Recent advances in AI are said to make programming obsolete because, given an English-language prompt, the newest systems can produce code for many commonly implemented algorithms. Adherents of this position, in contradistinction to Kemeny and Minsky, argue that programming has been automated and so it need no longer be taught to people. Many today see this as the promise of the newest forms of AI, but this view was already prominently expressed in 1962 when Peter Elias, then the head of the Electrical Engineering and Computer Science Department at MIT, argued that in “the next ten years . . . there will be very little left, if anything, to the teaching of programming” (1962, 203). In Minsky’s terms this is like arguing that no one needs to learn to play a musical instrument because most people have access to recorded music.

This position—music is for consuming, not for making, and programming is for industrial production, not for expression—reaches its nadir in what has now (2025) become an act familiar to most teachers. When a student hands in an essay produced by generative AI and disingenuously represents it as their own, it is an act of defiance, indifference, and self-harm as poignant as the declarations of Maurice Sendak’s character Pierre whose reply to every proposal is “I don’t care,” right up to and including the hungry lion who asks him if he’d like to die and—receiving the same response—eats poor Pierre (Sendak 1991). Delegating to an AI one’s opportunity to learn to express oneself in programs or prose or pictures is, today, an abject act of surrender and submission to the large tech firms that control the AIs.

Programs for People

Kemeny, Minsky, and many others conceptualize learning to program as learning to express oneself in a new language or a new medium. A textbook by Harold Abelson and Gerald Sussman crystallized this school of thought when they noted that “we want to establish the idea that a computer language is not just a way of getting a computer to perform operations but rather that it is a novel formal medium for expressing ideas about methodology. Thus, programs must be written for people to read, and only incidentally for machines to execute” (2022, 30–31). One of the cofounders of computer science, Turing Award winner Donald Knuth (1984/1991), Professor Emeritus of the Art of Computer Programming at Stanford University, articulated an analogous perspective when he wrote: “Let us change our traditional attitude to the construction of programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do. . . . The practitioner of literate programming can be regarded as an essayist, whose main concern is with exposition and excellence of style” (99).

Surveying Kemeny, Minsky, Abelson, Sussman, Seymour Papert, and other computer scientists who have argued for everyone to learn how to program, Annette Vee, in her book Coding Literacy, argues that “the roles that literacy has always played in our lives—the acquisition and communication of information, the central role in citizenship and education—are now changed by computers, code, and programming. . . . Computer programming is re-coding literacy” (2017, 457). Vee points out that (usually) implicitly, these computer scientists are advancing an argument about learning that situates computer programming in the longer history of literacy.

LLMs Are Not Models

Contemporary AI systems, such as large language models (LLMs) are—ironically—the polar opposite of what these founders of computer science and artificial intelligence wanted. Based on big data, they are “black boxes” impossible for any person to read much less understand. Typically their composition as computer programs includes billions of parameters. Consequently, even if we can open these black boxes (as one can with open-source LLMs, like Meta’s Llama, that currently [2024] comes in four sizes, the “lightweight” models with one billion or three billion parameters, the larger, “multimodal” with eleven or ninety billion parameters), we do not have enough time to read much less to understand the programs. Imagine trying to read eleven billion parameters and spending only one second per parameter. It would take over three hundred years, nonstop, to read all eleven billion parameters. Contemporary AI systems are not just large but sublimely large and certainly not human scale and so could never be “written for people to read” or taken as an explanation for human beings.

In the early days of AI, it was unimaginable that a computer program-as-theory-or-model could be too big to read. It was taken for granted that the text of a program would be readable, and so some concluded that the test of an AI program should not be readability but performance. Cofounders of computer science and artificial intelligence and Turing Award winners Herbert Simon and Allen Newell articulated the criterion of performance like this, in 1972: “A good information processing theory of a good human chess player can play good chess; a good theory of how humans create novels will create novels” (1972, 11). Certainly the LLMs of today write much better prose than any symbolic AI system that preceded them. But they fail as models and they fail as theories. Models (Latin “modulus”) are meant to be measures or standards (e.g., “model student”). Theories are principles used to explain phenomenon. LLMs cannot be models or theories because they are simply too big to be read, much less understood, by a human.

LLMs as Compressed Data

What then are LLMs if not models or theories? In the introduction to his autobiography, Models of My Life, Herbert Simon writes, again using novelists as his example, “it is the business of the novelist . . . to give us the data; it is up to readers to induce the theory from them” (1996, xv). Elsewhere, I argue that the “models” of contemporary machine learning are not theories but just compressed data (Sack 2019, 172).

A technical substantiation of this argument would examine how Soviet mathematician Andrey Kolmogorov (1963) articulated a theory of complexity employing a theorem from Ray Solomonoff’s 1960 publication “A Preliminary Report on a General Theory of Inductive Inference.” The Kolmogorov complexity of a collection of data is the length of the shortest computer program that can generate the data as output. We can understand the shortest computer program as a compressed form of the data. For example, consider data of two dimensions that mostly fall on a line. The data could consist of millions of points. Yet, the shortest program that could regenerate almost all of the data would be one that simply implements the algebraic equation for a line: y = ax + b. Any subset of the data that fell off the line (“outliers”) would be lost in its compressed form, in the computer program that simplifies all the data as a line. We can see these losses as “compression artifacts,” like those we see when a lossy compression/decompression algorithm (a codec)—like JPEG—is used to size down an image.

Contemporary machine learning algorithms “compress” big data into equations that are smaller than the original data, but still too large for human comprehension. While the algebraic equation for a line has two parameters (a and b), the polynomials induced by contemporary machine learning algorithms typically contain billions of parameters. But even with billions of parameters, the derived equation is smaller than the original data and so some of the irregularities in the data are lost. Consequently, when the derived equations are “reversed,” that is, employed to generate data (e.g., texts or images or code), they produce output that contains compression artifacts—visible losses or additions to the original data—or what are often called “hallucinations” in journalism about contemporary AI. The idea that LLMs are not models but just compressed data was recently, informatively, and humorously described by the science fiction writer Ted Chiang in an article for the New Yorker magazine: “ChatGPT is a blurry JPEG of the web” (Chiang 2023).

Reports of the Death of Theory Are Greatly Exaggerated

In 2008, the journalist Chris Anderson, in an editorial titled “The End of Theory,” has insisted that productions of machine learning algorithms are neither theories nor models. He cites Peter Norvig, Google’s research director: “All models are wrong, and increasingly you can succeed without them. . . . We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot” (Anderson 2008). It is difficult to see ourselves as named by what Anderson means when he says “we,” or when Norvig says “you.” “We” and “you” seem to be exclusively the data-rich. Arguably Anderson and Norvig are not referring to people at all but only to large corporate entities and powerful government agencies.

The rest of us (essentially all human beings), unlike Google, will have to settle for models and theories. And since we do not have access to “the biggest computing clusters the world has ever seen,” our models will need to be sized considerably smaller than contemporary LLMs. Norvig admits that these AI “models” of big data are not at human scale: “I agree that it can be difficult to make sense of a model containing billions of parameters. Certainly a human can’t understand such a model by inspecting the values of each parameter individually” (Norvig 2011).

One might argue that we learn all kinds of things (especially physical skills) without an explicit theory. For example, professional basketball players do not necessarily have a theory of basketball. Nevertheless, athletes do have models, people who have shown, through demonstration and emulation, how to do the sport—coaches, older siblings, better players. Essential, however, for this kind of model-based learning is that the model be a human whom one can emulate.

LLMs are not small enough to be theories (because they are too big to be read) nor are they performative in a manner that would allow them to be models for a person to emulate. Perhaps LLMs can be models or theories for corporations (at least in the United States where the Fourteenth Amendment provides corporations the status of juridical personhood) but cannot be the same for those of us who are not corporations.

AI and Epistemic Risk

While much ink has been spilled on the “existential threat” of AI, practically none has gone to addressing what seems to be a much more realistic fear. The epistemic threat of AI is the industrialization of knowledge into a form—like LLMs—that is only accessible to the biggest corporations and governments, a form that is not human scale. This combined with the indifference of students and experts alike to the prospect of outsourcing their skills and knowledge to AI systems portends a future in which no one knows anything nor how to do anything anymore. Admittedly, this sounds a lot like Plato’s fear (expressed in his Phaedrus dialogue) that the increasing use of writing would be the death of (oral) memory, but this epistemic risk is different. It is different because if the education system forgoes the responsibility to make sure everyone learns to program, the knowledge of the future will not be under the purview of human writers and programmers, but the exclusive possession of things, specifically the AI systems and the corporations and governments that control them.

References