1. Endgame Philosophy

BOSTROM’S FOUNDATIONAL 2002 ESSAY on existential risk begins by setting the bar incredibly and unnecessarily high for a risk to be existential according to his “typology of risk.”1 The definitional standard for a danger to be existential is “one where an adverse outcome would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential” (ER, 2). Bostrom claims that a risk needs to be both global and terminal to constitute an existential risk. In this model, “global endurable risks” that would create mass suffering but not lead to human extinction do not constitute true existential risks. In Bostrom’s typology, planetary-scale historically violent events are consigned to the less risky category of “catastrophe.” A terminal existential event means either total extinction or some kind of irreversible change that would structurally prevent humans from achieving their collective potential. In this classification system, genocide is by definition not an existential risk since it is only supposedly localized at the level of the “genos” or a kind of human life, not the whole of humanity. According to Bostrom, “An example of a local terminal risk would be genocide leading to the annihilation of a people (this happened to several Indian [sic] nations)” (ER, 2). World wars, the enslavement of vast proportions of the world, and bacterial and viral epidemics like the black plague and AIDS (and COVID-19) also would not register in existential terms since neither the entirety of humanity nor the whole structure that facilitates human potential or “Earth-originating intelligence” are at risk. As Bostrom states, “Tragic as such events are to the people immediately affected, in the big picture of things—from the perspective of humankind as a whole—even the worst of these catastrophes are mere ripples on the surface of the great sea of life” (ER, 2).

Here we pause already on the first pages of Bostrom’s essay to examine this controversial position. Perhaps the most startling aspect of Bostrom’s initial essay on existential risk is not its wide-eyed openness to any species-wide crisis, real or speculative, but his insistence on categorizing the large-scale violent events of the past and present as not at all existential threats. By demarcating only extreme calamity to be truly existential in scale, Bostrom’s claims for a strict existential threshold not only evinces obvious callousness, it sets the bar too high to be a useful measure. This model regretfully but purposefully ignores the violence and suffering inflicted on groups of people in which their existential condition is at stake. Bostrom assumes this violence has been and will continue to be contained as “endurable global risks since humanity could eventually recover” (2). This is dangerously close to rationalizing epochal histories of the suffering of minoritized and oppressed peoples for the sake of purported definitional consistency since something of “humanity” would survive. Furthermore, the threat of genocide and planet-sweeping violence still hangs over everyone and thus does bear directly on the existential condition of our species. While it may be the case that humanity as a whole is not imperiled immediately in the event of a particular genocide, that humans can intentionally commit such acts with explicit declarations that some lives are worth less than others indicates that there is nothing in the human condition that prevents humans from utterly destroying each other’s humanity. Does it really need to be said that the “logic” of genocide qualifies as extinctionary, even if the results are often “incomplete”? Why has Bostrom then not built this recognition into his philosophy?

Genocides do precipitate species-wide existential threats because those who perpetrate them seek a permanent remaking of the human condition, removing some ways of being human from the earth and installing a new way of being human according to new rules. Even that description fits Bostrom’s definition of an existential risk that would lead to a permanent constraint on humanity’s potential. As Hannah Arendt detailed, some humans can have their entire political being, their very access to the political, permanently removed from them. To lose one’s political being is to lose one’s existential condition. “What totalitarian ideologies therefore aim at is not the transformation of the outside world or the revolutionizing transmutation of society, but the transformation of human nature itself. The concentration camps are the laboratories where changes in human nature are tested.”2 There are no permanent, fail-safe solutions—political or technological—to prevent this kind of extreme violence used in transforming human nature by genocidal means. There is no political system to get everyone to agree and eschew violence, and there is also no existence without politics. Attempts to “solve” existential risks once and for all and institute a predictable and controllable definition of the human (in effect abolishing the political) are precisely the origin of genocidal logics. Since humans are inevitably risky toward each other, Arendt adamantly insisted that building durable public institutions, constructing worlds in common, and aspiring to shared long-term public goods are the only way to mitigate the inherent vulnerabilities of the human condition.

In many of his essays on existential risk, Bostrom has often claimed that there is very little academic research being done on this topic. In a 2013 essay, Bostrom remarks that a search of the SCOPUS database delivers 900 papers on dung beetles but the search category of “human extinction” registers fewer than 50 papers.3 “It is striking how little academic attention these issues have received compared to other topics that are less important,” Bostrom concludes. We did a similar search in March 2020 and found 100 papers on “human extinction” and 2,103 on “dung beetles”—but we also found 7,166 on “genocide” and 138,280 on “extinction” (though these numbers are inflated somewhat since “extinction” is a common term in psychology to denote the cessation of a stimulus, a denotation not relevant to existential risk). In JSTOR, a database primarily of humanities and social science research publications, the search “human extinction” delivers 66,809 results, “genocide” has 43,926 results, and “extinction” has 134,089 results. Similar evidence for the huge amount of research is also found if searching for specific existential risks like nuclear war, pandemics, or genetically engineered bioweapons. Both databases deliver a little over 100 results for “existential risk,” but the point is that Bostrom remains too tied to his own selective terms and criteria for what counts as truly existential.

Let’s turn the tables here. There is a stunning lack of attention in existential risk studies to the huge amount of research, activism, and human rights work on the history and prevention of genocides. The technocratic outlook and terminological narrowness of Bostrom’s assessments are partly at fault, but more disconcerting is the way his work ends up disclosing a colonialist attitude that downplays the history of genocides and Indigenous suffering. There is no evidence that Bostrom (or really any scholar associated with the burgeoning existential risk movement) has informed himself of the multitudinous Indigenous responses to genocide. Bostrom also insists that anthropogenic existential risks only began in 1945, with the invention of nuclear weapons, indicating the first time that humans could wipe out the whole of humanity. The Potawatomi scholar Kyle Powys Whyte points out how the privileging of the current moment as truly unprecedented in terms of its potential apocalyptic violence relies on a strategic distancing from past genocidal violence and also implicitly diminishes the knowledge of survival long cultivated by global Indigenous communities. Whyte remarks, “The hardships many non-Indigenous people dread most of the climate crisis are ones that Indigenous peoples have endured already due to different forms of colonialism: ecosystem collapse, species loss, economic crash, drastic relocation, and cultural disintegration.”4 Whyte adds that Indigenous peoples already see themselves as post-apocalyptic and navigating longstanding ongoing crises, connecting post-1945 conditions to the longer history of colonial ultraviolence.

The Terminal Scale

Bostrom’s setting the bar for existential risk at apocalyptic levels would make it so that the only event to qualify would be verifiable only after everyone is already gone and no one is left to measure it. This model in effect excludes all existential events in recorded human history, since none of these have been global and terminal. We also find in this argument that taking only the totality of the human species (and perhaps the future transhuman) as the norm of analysis directs the field toward considering as truly existential all-or-nothing visions of sweeping change only at the vastest scales. As Alaimo points out, this abstracted Anthropocene perspective of the human species as a whole works by “scaling up so that human poverty, drought, flooding, or displacement is obscured from sight and the viewer is not implicated.”5 The problems of scale bound up with individual and local group thought and action in connection to global phenomena—the multitiered scale of the existential, embodied subject embedded in a planetary commons—are swept up in this disembodied species-level overview. The medium level of politics, social activism, intersubjective care, and institutional building does factor occasionally into Bostrom’s thought, but mostly when he notes that anything done at this level to mitigate existential risk would likely have little effect in stemming most of the dire disasters facing humanity.

In response to Bostrom, most of the scholars in this growing field of existential risk have preferred not to insist exclusively on a standard of total extinction and have steered the discussion toward the broader category of “global catastrophic risks” (and Bostrom himself coedited a book of the same title6). Bostrom subsequently has incorporated the broader category of catastrophe into his work but retains his insistence that fully existential risks must be “pan-generational” (affecting all future generations) and go beyond the “endurable” toward the “crushing.”7 He thus remains committed to the view that there are really two main categories of existential risk—human extinction and anything that prevents humanity from reaching a transhuman condition. This position still remains prominent in the field of existential risk, as evidenced, for example, in Phil Torres’s definition of existential risk as “any future event that permanently prevents us from exploiting a large portion of our cosmic endowment of negentropy to create astronomical amounts of those things that we find valuable.”8 Toby Ord also follows along similar lines, including the supposed “plateauing” of humanity as an existential failure in his central definition: “An existential risk is a risk that threatens the destruction of humanity’s longterm potential.”9

As we will see, the most common effect of this definition is to convince that glorious posthuman futures are not to be seen as merely possible but rather obligatory—we must make good on these astronomical long-term potentials, anything else is a catastrophic failure. Fear of disappointing this big Other, the imaginary great future of humanity, runs rampant across the field. Such imperatives make it possible to understand transhumanism in the context of messianic history—especially Gnosticism and the Christian narrative of redemption.10 Though it is not our focus in this book, the ostensibly secular and scientific arguments of transhumanism, especially when they push for the disembodiment of mind into AI, defer to deep-seated patterns of religious thought.11

Typological Thought

The bulk of Bostrom’s 2002 essay is devoted to the classification of existential risks. He launches into this project by proposing four idiomatic classificatory categories: “Bangs” (a sudden extinction), “Crunches” (human civilization is stunted and humans never achieve transhumanity), “Shrieks” (only a narrow transhumanity is attained), and “Whimpers” (some transhumanity is achieved but it also impoverishes things we value). These remarkable terms are inspired by T. S. Eliot’s “The Hollow Men” (1925). We are left to wonder if these categories are forged by philosophical fiat, or by scientific consensus, or posited as metaphorical guidelines. Yet how should one construe the overlap of the methodological contributions of science and the literary imagination? We have no problem with employing metaphorical speculations to aid in thinking extinction, but why these metaphors? Are they the only categories plausible or available?

Before we look closer at Bostrom’s list of existential risks, this is the right moment to pause and reflect on the kind of philosophical style and its effects on argumentation that Bostrom most often employs in this essay and many subsequent others. While his philosophical principles are primarily drawn from utilitarian rationalism, risk management, and speculative transhumanism, he generally organizes his essays by providing typologies and lists, a writing style that has its own philosophical effects. The main advantage of this presentation style is that he delivers an overview of existential risks with clarity and concision. Readers can quickly assess the looming existential threats at a glance, and it feels like one is reading a kind of high-level governmental briefing. The disadvantage is that this presentation style can shortcut further philosophical analysis and critique. The brevity and directness of lists and typologies also means that many of his philosophical assumptions proceed unmarked and unargued. What kind of philosophy is typological thinking? Typological lists offer little if any forum for discussion and extended reflection. Typological thought tends to dispense with any extensive citation or broader background reading. It suggests that the issues already have been distilled to bullet points to summarize what we need to know in order to cognize existential risks.

Actually, Bostrom cites very few philosophers from any stripe or school in these essays. Interpreting this absence, it would seem that Bostrom does not find that traditional philosophy has much to offer in extreme existential situations. Is philosophy too human, too discursive, too argumentative, too slow, too powerless in the face of tremendous technological developments and planetary catastrophes? Bostrom does not dwell on the problems of philosophy itself. Yet his philosophical writing is largely a rejection of the philosophical discipline, an admission that philosophy has not done enough to make the world better or safer, and that it would be more efficient and effective to “align” philosophy with technologies to do the work that philosophy alone had tried but failed to accomplish. Instead of philosophy realizing a world of enlightened reason, superintelligent technology will have to do it.

Typological thought slants analysis away from much of the humanistic toolkit of philosophy and toward top-down policy propositions and heroic technocrats, implying that these are the true agents who will protect humanity from the disastrous future. Such typologies tend to be decontextualized and ahistorical pronouncements that rush the reader toward immediate judgment—“Here is a list of doomsday scenarios, there’s no time for debate or dallying, we must act now!” The existential risks that Bostrom lays out are serious and we do not mean to belittle Bostrom’s clear commitment to encourage further research from a wide variety of approaches to analyzing existential risk. We are concerned rather with how Bostrom’s own stylistic reductiveness shapes his argument as well as the trend of argumentation across the field.

Finally, on the topic of typological philosophy, Bostrom’s philosophical style does breed a tremendous amount of jargon. The language of bangs, crunches, shrieks, and whimpers is a clear example,12 but even more basic terms function as jargon. Terms such as “existential,” “risk,” and “global public goods” (ER, 4), along with categories of existential risks themselves such as “dysgenic pressures” (ER, 11) and “technological arrest” (ER, 12), become expedient insider terms used without reflective analysis. To be sure, all philosophies have jargon. There is nothing wrong with using specialized terms. We employ plenty of jargon in this book—we’re not shy about it. Our point here is that jargon can be mitigated with a commitment to building self-critique and a process of social mediation and public responsibility into the methodology of one’s analysis. Marginalizing these practices of critique, sharing, and consultation leads Bostrom to confirm his own view that these longstanding humanist modes of inquiry used to build the public sphere will be of little use in confronting the overwhelming violence of existential risks when they do occur.

Even disasters breed jargon. One prescient example of the brutal effect of this can be found in Naomi Klein’s analysis of disaster capitalism and the mobilization of neoliberal textbook language into schemes to transform national public services into open markets in The Shock Doctrine.13 Bostrom’s “apocalypse now” methodology has little interest in examining how the effects of catastrophes are unevenly distributed, exacerbating existing social inequalities, and further entrenching economic disparities and dispossession among peoples and between nations.

It bears recalling here Theodor Adorno’s trenchant study of what he called the “jargon of authenticity”—a critique Adorno specifically launched against the proliferation of jargon in existential thought, especially in Martin Heidegger’s version. Adorno points to how quickly even the term “humanity” can become jargon when its invocation is used to deflect from any specific devotion to social betterment. When “humanity” is transformed into jargon, “Humanity becomes the most general and empty form of privilege.”14 While exclaiming this jargon, the human domination of other humans and nature continues unabated. The jargon of humanity, Adorno adds, “caricatures the equal rights of everything which bears a human face, since it hides from men the unalleviated discriminations of societal power: the differences between hunger and overabundance. . . . In the mask of jargon any self-interested action can give itself the air of public interest, of service to Man. Thus, nothing is done in any serious fashion to alleviate man’s suffering and need” (54).

One example of Bostrom turning “humanity” into this type of jargon comes in a 2013 essay where he supplies a graph of the rise of world population over the last century showing a slow, consistent rise of total population from 1900 to 1950, and a steeper rise (called the “great acceleration”) from 1950 to 2010. Bostrom provides the following caption as interpretation: “Calamities such as the Spanish flu pandemic, two world wars, and the Holocaust scarcely register. (If one stares hard at the graph, one can perhaps just barely make out a slightly temporary reduction in the rate of growth of the world population during these events).”15 The upward graph is taken as “proof” that these “calamities” were not really existential events. The generic category of humanity, now turned into a population data point, has survived and moved on. No other lessons are gleaned here for existential risk. Bostrom, of course, is not denying historic suffering; the point is that the graph is used to confirm Bostrom’s own methodology, concealing his controversial polemic about what truly constitutes an existential risk as simply “confirmed” by the data.

Beginning with the End

Turning back to Bostrom’s list of “bangs,” these potentially devastating events are the core of what is commonly understood as the extinction crisis facing humans. His category of “bangs” lists the following in descending order of probability (beginning with what Bostrom estimates to be most probable, although he assigns no specific risk calculations here): deliberate misuse of nanotechnology, nuclear holocaust, we are in a simulation (a cosmic algorithmic program run by some form of superintelligence) and it gets shut down, a superintelligence of our own creation leads to our undoing, a genetically engineered bioagent, an unforeseen physics disaster of our own creation (for example, a new particle that annihilates Earth into a vacuum), a naturally occurring pandemic, an asteroid impact, and runaway global warming (as likely occurred on Venus). Such lists exemplify the “rhetoric of probability” we discuss in chapter 2, where existential risk discourse takes these probabilities as a spur for speculation on dystopian or utopian outcomes rather than actually carrying out calculations. With hindsight of the coronavirus in 2020, it’s curious that pandemics and extreme global warming rank lowest in probability, although, as we know, Bostrom would not rank the current concerns as existential, merely catastrophic.

In compiling this “worst ever” list, Bostrom often proposes a utopian counterpoint to each dystopian scenario. The deliberate misuse of nanotechnology follows along the same route as the utopian possibilities of this technology. Things could go very badly in many ways, but, “If things go well, we may one day run up against fundamental physical limits. . . . But here we are talking astronomical time scales” (ER, 14). Extinction looms, but so do the most immense successes imaginable. Indeed, for Bostrom, the failure to eventually achieve the utopia of “astronomical” value in the long run and leave the human condition behind will always be dystopian. Bostrom keeps insisting that it would be an existential risk to not pursue utopia (a position we will scrutinize further below), by limiting ourselves to an “extremely narrow band of what is possible and desirable” (ER, 5). This oscillation between dystopia/utopia continually appears in Bostrom’s argument. It parallels his seesaw of claims that we have various biases that tend to underestimate some existential risks and overestimate others, suggesting we will always undershoot or overshoot a problem.

To his credit, Bostrom’s foundational 2002 essay intriguingly fosters an openness toward science fiction–like philosophical “what if” scenarios of planetary catastrophe. In another essay drafted just prior to the “Existential Risks” essay, Bostrom walked through some speculative arguments about whether or not it might be the case that humans are living in a computer simulation run by an exponentially higher intelligence.16 If we are living in virtual cosmos, it would constitute our ultimate existential risk since the simulation could be shut down at any time. Extinction would have an entirely different meaning than the biological definition we assume right now. This provocation that extinction might be something radically different raises other far-reaching problems. If life is substrate-neutral and can appear in different underlying mediums, materials, and programs—or if life is implicit in all matter (panvitalism) like mind is for panpsychists—then extinction and existential risk will require massive redefinition, pushing the concepts of extinction and risk beyond coherence.17 Methodologically, Bostrom’s range of hypothetical existential risk thinking has fostered an open-minded inquiry into any conceivable end of the world. However, as we discuss further in chapter 2, the philosophers of existential risk do not want their work confusing science and science fiction, and rationalist utilitarian and probabilistic thinking does not methodologically align with speculative metaphysics (or existential thinking, for that matter). The field requires a coherence to the definition of existential risk and extinction, but these methods of reasoning and calculating probability run into a number of inconsistencies and incoherencies.

Bostrom’s other categories of “crunches,” “shrieks,” and “whimpers” carry on in the same mode, mixing scientifically evident problems with hypothetical scenarios to classify near-term planetary disasters and long-term speculative social, biological, and technological bottleneck phenomena (the notion that we have to squeeze through a very risky historical period to get to a more stable and advanced society). Since Bostrom is supplying a brief bird’s-eye view of existential risks, he does not bother to incorporate much in the way of a critique of existing social inequalities. While resource depletion and ecological destruction do rank as possible “crunches,” the unexamined economic and social arrangements that underlie these risks include capitalism, colonialism, and any economic system that requires endless growth to function. Moreover, we find no analysis of the fact that the pursuit of new technologies combined with repressive political regimes will exacerbate the suffering of those who already find their lives exposed to a high degree of existential risk across the planet. Kathryn Yusoff remarks that Anthropocene discourse of the generic human species ends up effacing how material infrastructures and resource extraction follow on the same tracks of colonialism and racialized power. The super-technologies and superintelligence that will save “us” will have to run on the minerals and extractive resources taken predominantly across the globe from Brown and Black communities and nations that historically have been treated as existentially expendable. Yusoff writes, “To be included in the ‘we’ of the Anthropocene is to be silenced by a claim of universalism that fails to notice its subjugations.”18 While some are existentially secure—and can raise inquiry into how such security is possible—others have long been exposed to everyday existential insecurity. Bostrom’s risk scenarios have no specific insight into how this structurally uneven distribution of existential risks also produces existential privileges.

Bostrom’s expanded range of “what if” scenarios that cover immediate concerns as well as hypothetical long-range possibilities can make it harder to assess and prioritize more urgent and likely calamities. For example, while he mentions near-term concerns over environmental crises, they are not particularly highlighted or examined in the context of any broader environmentalist commitments. Bostrom points to the risk of greenhouse gases on Earth leading to runaway global warming (ER, 10) such as found on Venus—an ecological problem, to be sure, but one pitched at the level of an improbable total destruction of habitable life on a cooked Earth and not particularly engaged with the everyday work of ecological care. Another environmental existential risk mentioned is the depletion of primary resources (nonrenewable fuels, timber, water) leading to civilizational collapse. These limited resources are certainly immediate concerns, but Bostrom and others in the existential risk field who tend to rely on typological lists of disaster scenarios have not prioritized the nonspectacular, hard work of researching water protection or divestment from oil and coal and the collaborative energy transition to come. Bostrom tends to keep his eyes peeled for quick and dramatic risks. He offers only occasional nods to what Rob Nixon calls the “slow violence” of accumulating environmental degradations that have become features of daily life for the global poor.19 Because existential risk examines only the extremes of human suffering, there is little interest in small-scale fixes.

Here we can point out that all of Bostrom’s existential risks focus strictly on humans and have nothing to say regarding extinction threats to animals. Bostrom does not show much interest in animal life beyond a rudimentary utilitarian concern to reduce suffering across species. So far, none in the field of existential risk seem to have applied the analysis of human existential risks with regard to animals or studied how the history of animal extinction informs human extinctions. Winona LaDuke shows that the continued elimination of Indigenous peoples intersects with the ongoing destruction of animal and plant species: “There have been more species lost in the past 150 years than since the Ice Age. During the same time, Indigenous peoples have been disappearing from the face of the earth. Over 2,000 nations of Indigenous peoples have gone extinct in the western hemisphere, and one nation disappears from the Amazon rainforest every year. There is a direct relationship between the loss of cultural diversity and the loss of biodiversity.”20 The diversity of peoples and languages mirrors the diversity of species and the disappearance of peoples historically has coincided with the disappearance of the planet’s biodiverse life.

Doom Patrol

Bostrom’s essay details some reasons why existential risks pose a unique set of challenges for human responsibility and traditional moral reasoning. He suggests that the magnitude of risk is so large that conventional use of trial-and-error to correct problems is insufficient. For some existential risks, especially new ultrapowerful technologies and weapons, we may only have one chance to get it right. The first person or group to achieve such technologies and weapons may have the power to dominate or destroy everyone. Everything depends on being able to shift what might be an existential risk into, if not full prevention, a “mere” catastrophe that would allow us time to have more than one chance to act—time to learn and teach ourselves how to live together again. Bostrom’s estimation that we only get one chance can also be questioned as it assumes a fully formed AI or doomsday weapon functional as if overnight. Bostrom argues that there may be situations where some kind of “preventive action” (ER, 3) is needed, a potentially militaristic claim akin to justifications for “pre-emptive” war-making. Bostrom also airs the concern that any current institutions, state actors, or moral norms would be insufficient to the task of avoiding or managing existential risks. The lack of precedent, the likelihood of panic, and the probability that we underestimate such violence and overestimate the time and skill we have in solving terminal planetary problems are all factors suggesting current norms will not abide at existential threat levels. A truly existential event would induce a radical “state of exception,” one in which there may even no longer be a state or identifiable political authority.

Relying on human capacities for responsibility in exceptional events presents other risks, and many ethicists argue that a more scientific and “objective” approach to ethics may involve taking ethics out of human hands, favoring institutional or technological decision-making. Existential risk scholar Phil Torres rightly emphasizes vigilance toward what he calls “agent-tool couplings,”21 situations in which powerful tools can fall into the hands of bad agents. Among bad agents, Torres lists dictators, misanthropists, and believers in apocalyptic violence as redemptive according to eschatological religious narratives. Torres also adds “ecoterrorists” and other “misguided ethicists” who might want to erase humans from the Earth. However, how is this vigilance able to make distinctions between real people who strive to eradicate all of humanity on a moment’s notice and discursive statements and performances of the apocalyptic imagination, however nihilist or ecofascist? In any case, one massive problem with Torres’s list is that any agent with a grudge against “humanity” becomes a risk that supposedly justifies using “preventive action” to subdue. Yet agents who use such preventive action also pose new risks of their own to the planet by calling for mass surveillance and incarceration as the only apparent way to weed out malicious agents before they strike. A situation could arise in which there are multiple agents claiming different kinds of emergencies that prioritize different kinds of existential risks and use extrajudicial and preemptive methods to destroy what they deem as nefarious agent–tool couplings. The violent history of this kind of totalitarian paranoid power is obvious and the existential risk theorists who find themselves airing these scenarios as unsavory but perhaps necessary need to answer to why their methodology offers little else in terms of “foresight.”

Despite these misgivings and doubts on institutional resilience and individual responsibility, many of the researchers involved in this field have concentrated their advocacy efforts on global governance and scientific collaboration. Bostrom and others continue to support notions of sustainable good governance in ecological, technological, and economic systems, though these systems may not always be compatible and may still fail in an extreme event. Although the need for planetary governance would seem to point to a discussion of collectivism or planetary political commons, researchers in existential risk tend to look askance at any form of radical political change or unified social horizon. They are not calling for a movement to defund the military even as they advocate for nuclear disarmament, although the contribution such funding makes to the proliferation of weapons surely constitutes an existential risk (and yet, military programs like DARPA are major funders of superintelligence research). They are not leading researchers into the recent waves of white nationalism and authoritarianism, movements that have articulated a vision of remaking the planet through violence, xenophobia, and oppression (and here it bears mentioning that a number of libertarian transhumanists have quietly supported attitudes of white escapism).

Perhaps most conspicuously, the field of existential risk offers no substantial critique of capitalism, in which an indefinite ongoing demand for growth and resources poses an eventual existential crisis (historically, economic risk and capitalism are synonyms indicative of the spread of financial risk and reward across social classes22). Existential risk researchers seem to prefer neoliberal practices setting the current standards of value according to markets regulated lightly along utilitarian lines (interest in universal basic income is not touted as a way to abolish exploitative capitalism and extractivism). In Risk Criticism, Molly Wallace describes how the “outsourcing of risk” has been built into capitalism in which nations and corporations “play the unevenness of the global market by capitalizing on the gaps in risk regulation.”23 Often the worst realities of these risks are transferred onto disenfranchised peoples who have a long and brutal road to haul for redress. Truly dampening the recurrent recklessness of economic risk would have to involve transitioning out of capitalism. But it seems to be easier to imagine changing the existential structure of humanity than to imagine the end of capitalism. Indeed, capitalism now looks to existentiality as its new frontier, taking on the core list of traditional existential concerns—freedom, moods, intentionality, natality, and mortality—as markets for engineering and programming.

Endgame Moves

Bostrom closes his initial essay with a “rule of thumb for moral action” that he dubs “maxipok”: “Maximize the probability of an okay outcome, where an ‘okay outcome’ is any outcome that avoids existential disaster” (ER, 25). Hardly stirring, this instruction is vague and directionless, closer to the Silicon Valley dictum of “don’t be evil” than a clearly defined new categorical imperative. But the jargonistic phrasing of “maxipok” proves to be deceptive, since Bostrom makes it clear that he does not condone just any outcome for humanity. In contrast to this seemingly nonchalant expectation for an “okay outcome,” Bostrom insists in the essay that “technological arrest,” whereby humans never “transition to the posthuman world” (ER, 12), would constitute its own kind of existential failure. To just be okay is not okay. Built into Bostrom’s philosophy is the argument that to remain existentially the same (merely “okay”) and not achieve a posthuman triumph would be an ultimate failure.

The audacity of arguing that merely remaining human would be a kind of existential threat boggles the human mind. In an essay drafted in 2007, Bostrom outlines four primary possible human futures: extinction, recurrent collapse, plateau, and posthumanity.24 In Bostrom’s thinking, only the last category would be a worthy objective. The essay defines “recurrent collapses” as cycles of violence and assorted disasters in which civilization is set backward, advances, and is set back again. “Plateauing” indicates humanity failing to keep up the tremendous rate of technological advances and health improvements brought about in the past few centuries. Humans would plateau if they are not able to live substantially longer lives, eradicate most causes of suffering, and develop cognitive improvements either biologically or artificially that are at least two standard deviations above current human maximums (ER, 20). The timeline for escaping the human plateau is less important than its eventual accomplishment, which for Bostrom constitutes the ultimate definition of the good.

If humans don’t commit in favor of pursuing technological greatness in the short or even long term, is this really “plateauing”? Picasso is said to have uttered upon seeing the Paleolithic-era cave drawings in Lascaux, “We have invented nothing.” These images still stir because we don’t dismiss this art as an order of aesthetic magnitude or “plateau” below us. Aesthetics and ethics facilitate capacities that are cognitively enjoyable as well as enhancing; technological development is not the only barometer for expanded consciousness and experience. And it’s not as though we’ve explored the potential of what a much more widespread aesthetic and environmental education would do to society. Artificial intelligence or posthumanism might bring about a diminishment of experience in some ways, as computers process more of the world with or without us. There is also ongoing evolution and disruption in a plateau, and more to come in Deleuze and Guattari’s “a thousand plateaus”! The “plateau” is a metaphor that points to other dynamic metaphors of creative commons—consider then what futures for humanity arise by foregrounding the unruly garden, spaceship Earth, or another figure for the existential planetary journey?

2. Probability and Speculation