Introduction: What Is Existential Risk?

Existential Risk as a Philosophical Field

The phrase “existential risk” dates in its rise of popularity to Bostrom’s 2002 essay, “Existential Risks: Analyzing Human Extinction Scenarios and Related Hazards.” Bostrom’s essay, written in a clear, crisp, and brisk style, cuts through some of the bewildering and oracular rhetoric of apocalyptic thinking to make human extinction a topic for analytic and utilitarian philosophy. Using dispassionate prose, Bostrom eschews long-winded diatribes and fire-and-brimstone moralizing, presenting a brief survey of possible species-level threats. He offers a minimalist guideline for ethics in the face of such risks—thin on specifics and any self-critique. In a further remarkable move, Bostrom aligns the study of existential risk with a vision of the complete overcoming of existential problems, pointing to the apparent resolution of such risks in the potential “astronomical” value of massive technological transformations to come that would permit humans or Earth-originating intelligence to spread across the galaxy.8

When he wrote “Existential Risks,” Bostrom had been known primarily as an enthusiastic proponent of transhumanism.9 With the popularizing of the phrase “existential risk,” which he defines in his initial essay as “global, terminal risks” to humanity, Bostrom soon garnered widespread acclaim, leading to his professorship at Oxford in 2003 and his directorship of the Future of Humanity Institute in 2005.10 In 2012, Cambridge University established the Centre for the Study of Existential Risk, and a number of other institutions have followed, with significant private donor backing from technocratic elites such as Elon Musk.11 The Future of Humanity Institute received 13.3 million pounds in funding from the Open Philanthropy Project in 2018, an effective altruism group primarily funded by Dustin Moskovitz, who cofounded Facebook with Mark Zuckerberg and three others. Skype cofounder Jaan Tallinn is also cofounder of the Boston-area Future of Life Institute, a think tank with ties to MIT devoted to studying AI safety and other technological extinction risks. Musk sits on the FLI scientific advisory board, and on a similar board at the Cambridge Centre. Support for existential risk studies from deep pockets has become part of the story of the field. For us, what is important is how such extraordinary funding affects the methods and vision of the philosophers in the field, who consistently focus their attention on wondrous technological feats as the antidote to existential problems and generally eschew any concrete social or environmental justice initiatives, including those that contest the tech corporations hungry for the resources and energies that fuel the artificial intelligence boom.

As it stands, we might say that there are three categories of research on existential risk: (1) The self-conscious members of the field, a comparatively small but well-funded group of academics trained in analytic philosophy and risk modeling (along with a handful of scientists such as Rees and Max Tegmark). This is where the field has more depth than breadth. (2) The wider, popular media circuits in which the specialist claims of existential risk analysis circulate, which include a broad range of influential magazines and news sites. Outside its academic publications, the field has wide and enviable impact, but not so much depth. (3) The work of researchers, mostly scientists and data analysts, who publish things that thinkers like Bostrom consider relevant to existential risk. Here there is undefinable breadth and depth, but because of how easy it is to make the case for relevance to a topic as expansive as existential risk, we cannot cover this third category in detail. To clarify, our critique is directed at the first two categories both because we think the field and its impact are problematic and because these are theoretically instructive case studies for environmental humanists interested in the methods and impacts of conceptualizing extinction.

Some of the exigency of our project is spurred by the widespread popularity of Bostrom’s launching of the field under his terms, but we are also wary about how existential risk theory has been conducive to a warm reception by a “tech bro” Silicon Valley audience. Existential risk discourse currently thrives amid high capitalist fever for AI and transhumanist life extension start-ups. Slavoj Žižek acutely articulates the Silicon Valley ideology whereby triumphing over the “impossible” becomes the paramount focus of every company, while improving just a little bit the social welfare of the nation, let alone the planet, is deemed truly inconceivable: “You want to raise taxes a little bit for the rich; they tell you it’s impossible [because we] lose competitivity. You want more money for health care, they tell you ‘impossible; this means a totalitarian state.’ There’s something wrong in the world, where you are promised to be immortal but cannot spend a little bit more for healthcare.”12 Instead of the hard work of building a safer world together through universalizing norms of respect, dignity, and consent across varied ways of existential flourishing and social solidarity, Bostrom offers a list of calamitous scenarios and transhuman rescue fantasies that aim to eventually pan out in “masters of the universe” powers. As it is currently constructed, the field of existential risk ultimately reinforces anthropocentric thinking, cheers the capitalist acceleration of AI (as long as it builds in moral safeguards), elevates individualistic egoic satisfactions as the truth of “astronomical” values and the destiny of technology, and flees at all costs from the existential commons of cross-species vulnerability and finitude.

The institutional history of existential risk thinking is quite banal in comparison to the dystopian feverishness and utopian bravado implicit in these arguments. But the references and style of argument in this field play a crucial role in determining what counts as an existential risk, who is most immediately exposed to life threats, and what “existential” means. It’s already a remarkable phenomenon to observe a new cross-disciplinary field being founded in its initial stages, but the stakes of this particular field are legitimately hair-raising and epochal. This field has not only turned some academics into major public figures, it has won them a seat at the table as consultants on some of the most important decisions being made by companies and governments that will affect everyone.

The researchers contributing to the field of existential risk, Bostrom included, have stressed adamantly the need for more scholarship and funding to study extinction scenarios and probabilities of civilizational collapse. Yet this field has cultivated a noticeable lack of diversity of references, voices, and participants so far, with existential risk scholars being mostly white males from wealthy Western countries and well-funded foundations and universities. These countries, institutions, and identities are not facing immediate existential risks. While we do not deny that all humans everywhere might face sudden extinction threats at any time, what stands out is how little attention existential risk theorists are paying to marginalized peoples and colonized existences that face the end of their way of life on a daily basis. Also troubling is the lack of engagement with the long history of research and testimony of those whose lives and cultures have been threatened with destruction. The vast majority of scholarship in the field of existential risks ignores reams of scholarship and personal accounts of living through mass suffering and extreme violence, preferring instead to jump to speculative and spectacular scenarios summed up in bite-sized terms and axioms.

Environmental Humanities and Risk Criticism

In parallel to the field of existential risk, new approaches to the study of extinction are growing in the humanities, emphasizing environmental justice and multispecies storytelling.13 The history of understanding extinction is tied to humanistic inquiry about how to live together and share a world and what happens when worlds in common are lost. Extinction is a biological fact and also conceptual crisis that can potentially put all of our concepts and values into question, because these too are at stake in the disappearance of ways of life. Extinction entails the existential end of a life form, while also compelling us to think about the existentiality of the terms we use to understand and respond to extinction. Examining the existentiality of concepts and terms involves considering how the methods we use run into their own limits and crises in thinking extinction. In the eloquent phrasing of Matthew Chrulew and Rick De Vos,

Extinction challenges our thinking and writing. Such overwhelming disappearance of ways of being, experiencing and making meaning in the world disrupts familiar categories and demands new modes of response. It requires that we trace multiple forms of both countable and intangible loss, the unravelling of social and ecological communities as a result of colonialism and capture, development and defaunation and other destructive processes. It brings forth new modes of commemoration and mourning, and new practices of archiving and survival. It calls for action in the absence of hope.14

Thinking extinction also entails registering the impact of extinction on how we think. One must find a way to address the end of address for whichever life form, including the human, might disappear. As we show, this incorporation of the finitude of thought into philosophy is a hallmark of existential thought. This paradox, using thought to imagine the end of thought, has produced a tradition of philosophical inquiry generative of new ways to imagine existential risks.

Understanding any extinction event requires attention to the specificities of embodiment, care, communal memory, power over life and death, collective mourning, evolutionary theory, and ecological connectedness that allow us to cognize both world building and world loss. Yet, as we will see, concepts like these that have become central to comprehending extinction in the environmental humanities have garnered little attention so far from the field of existential risk. Advocates of existential risk philosophy are preoccupied foremost with tallying a range of immediate and far-term doomsday scenarios. Thinkers in this field employ a version of philosophical rationalism, statistical analysis, and speculation on the future of risks to human survival. But these are not the only ways of understanding extinction risks, and not the only practices and discourses of existential urgency.

In what ways are existential risk and environmental humanities complimentary, and in what ways do they markedly diverge? How might both fields contribute to what Molly Wallace calls a “risk criticism”? Risk criticism combines both cultural and technological reflections on risk toward developing not just urgently needed foresight, but also a sense of “precautionary reading” that “emphasizes its uncertainness, unpredictability, and incalculability.”15 Precautionary reading links to Wendy Hui Kyong Chun’s argument that climate change disaster scenarios use “hypo-real” prediction—that is, a modeling between hypothesis and reality—to help avoid the outcome they predict. “If models work properly as evidence, they become unverifiable: if we are convinced of their verisimilitude, we will act in such a way that their predicted results can never be corroborated by experience.”16 In existential risk analysis, the primary aim is to come up with a projection in which uncertainty becomes a practical tool used to foreclose the disastrous futures it anticipates. “Precautionary” existential risk analysis requires the humanist tools of associative, critical, and creative speculative thinking as well as data gathering and rationalist rulemaking.

Existential Re-Mix

In yet another strange omission, none of the philosophers who are shaping the field reflect on the conceptual history of the term “existential.” They are existential thinkers without thinking existentialism. None seem to care that existential philosophy reached its intellectual peak precisely as a response to the existential risks of the first half of the twentieth century, forcing philosophy to reckon with scientific evidence of species extinction, world wars, genocides, concentration camps, nuclear weapons, imperialism, and structurally oppressive systems of racism and sexism. Existential thought still provides relevant and useful tools to reflect on existential risks. The methodological shortcomings of existential risk should not be excused in the name of epochal urgency. As we will argue, these preferences and limitations go to the heart of the stakes, reasons, and practical outcomes of thinking existential crises today.

We are not subscribers to any school of existentialism. Our claim is rather that the contributions of existential thought to the field of philosophy—specifically the emphasis on a need for both subjective and objective accounts of subjectivity and the becoming of knowledge, following Hegel’s phrasing that “everything turns on grasping and expressing the True, not only as Substance, but equally as Subject”17—provide crucial insights for comprehending existential risks. However, we do not define human existence on terms typically associated with existential-humanism. For us, the human is a “cybernetic triangle”18 of animal-human-machine, terms that both conjoin and mutually oppose each other and yet together compose the phenomena of human life intertwined with other kinds of existence.19 We offer an initial definition of (unaffiliated) existential thought based on three core claims: (1) Existential thought thinks the structural conditions and phenomenal “lived experience” of everyday existence; (2) existential thought thinks the limit-experiences of existence, that is, the extremes of existence, including mortality and extinction, as well as phase transitions from one form of existence to another; and (3) existential thought implicates the thinker in the thought, such that the participation of the thinker has an effect on the thought. We find that existential risk overwhelmingly borrows from the second definition while neglecting the first and the third. Yet the analysis of “lived experience” and embodiment as expansive for thought (not something subdued by probabilistic reasoning or superseded by enhanced intelligence) is central to what the term “existential” means and so also to our understanding of existential limit risks. Uncoupling one definition of “existential” from the others will have profound consequences, as we demonstrate later.

Bostrom makes it clear that he views the restriction or demise of intelligent life, not necessarily human life as such, as the utmost calamity. Bostrom prioritizes expanding “the space of reasons” toward “astronomical” values as the broadest horizon for his philosophy of existential risk. This view of expansive reasoning, superintelligence, and galactic-scale value premised on maximizing these apparent utilitarian goods shapes significantly how existential risks are defined and assessed. In Bostrom’s thinking, because existential risks are endemic to humanity and have no permanent solutions, we have a duty to not only expand our philosophical knowledge and rational discourse on existential risk but also pursue modifications to our existential condition. These modifications can range from enhancing intellectual, biological, and technological powers to improve our ability to mitigate existential risks and to maximize the utilitarian value of any number of future lives.

Existential thought is concerned with the philosophical basis for expanding the possibility and accessibility of world-making and world-sharing across the space of existences. In the face of rising existential risks and contestations over definitions of “existential” and “risk,” we reaffirm the existential condition for expanding ways of flourishing together and sharing existence on Earth, which aligns with the aims of environmental justice as “the search for fair ways of sharing environmental burdens and benefits and collectively creating a future in which the dignity and rights of all people are respected.”20 Expanding the space of reasons is not the only criterion for truth.21 Expanding the space of existence toward ends of environmental justice includes creating broader conditions of existential flourishing for diverse ways of life on a shared planet. The space of existence includes the range of embodied affects, experiences, material and ecological relations—these all contribute to expanding the space of thought and existentiality, which is not reducible to intelligence or utilitarian calculations of present and future value. To paraphrase Kant, reasons without existence are empty, while existence without reasons is blind.

What does this argument for expanding the space of existence have to do with existential risk? To summarize our reply:

1. All the existential risks facing humans and other life forms on Earth concern what Hannah Arendt called the fundamental necessity to “share the Earth.”22 In our view the existential condition is the ecological condition. Extinction and ecology are intertwined phenomena; extinction makes no sense without a context in which the singularity of a life form makes a unique and irreplaceable contribution to expanding the space of existences. There is no such thing as ecology without processes of natality and mortality, speciation and extinction, situated in specific habitats. Too much extinction destroys ecology, but no extinction at all is another way to eliminate the ecological condition. The concepts, methodology, and mitigation of existential risks need to build in this intertwined existential and ecological condition rather than seek to dismantle this connection in the name of some speculative immunity.
2. All decisions regarding the permanent change of the existential condition for humans or nonhuman animals should involve consultation and consent of all species existences. This is clearly an ideal scenario that is not currently feasible, but it should be built into the methodology of existential risk studies. We have no current political models for this form of radical politics. We have no form of address in which to address all existences. Yet any change to the human existential condition would involve all current and future humans, whether or not they consent, and would impact the vast majority of animal life as well. We will need to imagine and institute new forms of address, consent, respect, and inclusion—a process that none of the existing literature in the field of existential risk prioritizes.
3. Bostrom’s theorizations and remedies for existential risk divide up the present and future of humanity along the restrictive lines of a hierarchical valuation of intelligence. Bostrom claims that existential risks show us that enhanced intelligence toward a transhuman condition is not just a possibility and a favorable option, but a moral and ontological obligation. Bostrom consistently conflates the obligation to prevent human extinction with the obligation to develop humanity toward transhumanism. This conflation is predicated on an evaluative hierarchy of superintelligence that places other forms of embodied and ecological intelligence into subordinate positions (this is Claire Colebrook’s critique, discussed in chapter 3). In Bostrom’s view, humans can modify the animal–human–machine existential condition toward one that abandons the animal part and accelerates the human–machine complex toward a more promising future. Our claim is that the animal–human–machine condition functions as a package deal, such that expanding the space of intelligence is connected to expanding the space of existential flourishing. We doubt that it is either possible or desirable to peel existence away from the evolutionary trajectories and ecological interdependencies of life.
4. The existential condition is the basis of our elementary commonalities with all other life on Earth. Existential risk analysis should not be about abandoning, sterilizing, dominating, or solving once and for all the existential commons but finding ways to flourish together. Such existential commons are the basis for our intersubjective sociality with human and nonhuman life, what N. Katherine Hayles, writing on the novel coronavirus in 2020, has called “humans as species in common” and “humans in biosymbiosis.”23 Any change to this structure would reverberate across the planetary biosymbiotic commons, permanently changing those commons and perhaps revoking the conditions of possibility for having a shared planet at all. Our position aligns with what Deborah Bird Rose advocates as “ecological existentialism”24 and Stacy Alaimo’s call for an ethics and politics based on grappling with the exposure and entanglement of our bodies with other bodies. Alaimo writes, “Performing exposure as an ethical and political act means to reckon with—rather than disavow—such horrific events and to grapple with the particular entanglements of vulnerability and complicity that radiate from disasters. . . . To occupy exposure as insurgent vulnerability is to perform material rather than abstract alliances, and to inhabit a fraught sense of political agency that emerges from the perceived loss of boundaries and sovereignty.”25

Bostrom envisions a scenario in which existential risks eventually give way to the posthuman prosperity of enhanced cognitive capabilities. But the current existential structure we have already allows for different forms of first-person experiences, self-transformations, and collective flourishing. We are not rejecting Bostrom’s posthumanism out of a humanist fear of transformation or nostalgia for a simpler human condition, but with a sense of solidarity and collective possibility toward the future, which can include new technologies but need not rely on such technologies to save us. As Cary Wolfe and Zakiyyah Iman Jackson have argued, posthumanism can be a mutation of ideas and practices that disrupts existing racialized and speciesist norms of being human, not some effort to transcend our finite bodies and ecologies.26 We do not need any special new technology or enhancement in order to cultivate Rose’s “ecological existentialism” and Alaimo’s “insurgent vulnerability.”

The Three Critiques

Each of this book’s three chapters is a distinct critique of existential risk: a discursive analysis of the field’s foundational terms, a review of the use of science and probability theory from the stance of critical science and technology studies, and an examination of the existential implications of existential risk.

Chapter 1 looks at the language, methodology, and rationale of Bostrom’s first major essay on existential risk, published in 2002. This essay remains foundational for the field, especially in its terminology and broad overview of what constitutes an existential risk. We highlight in particular how Bostrom’s philosophical style makes heavy use of typologies and lists. Typologies are not forms conducive to philosophical debate and self-critical argumentation; they are brief summarizing statements designed to be read with speed. These typologies betray an impatience with traditional philosophizing, scholarship, and interpretive reading across the humanities and social sciences. We go into further detail examining how Bostrom constructs his list of existential risks, and how he reckons that existing institutional safeguards against such risks should be bolstered yet not really trusted. His conclusion that no current form of security would likely be adequate opens the door to authoritarian preventive actions and fantasies of total behavioral control, a position that bodes poorly for this philosophy and speaks to the problems at the roots of this field.

Chapter 2 provides a reflection on the use of risk analysis and probability thinking in the development of existential risk theory. This chapter draws on science and technology studies to critique the use of science, and the tendency toward scientism, in existential risk theory, while raising questions about the field’s relation to science fiction. Existential risk analysts seek to estimate the chance that a given cause will lead to the extinction of humanity or even life in general, using probability models that are generally Bayesian. However, these theorists also recognize that since there has never been a recorded existential risk event (extinction event) for humans, probability runs up against an impossible data point. The crux of existential risk analysis is the synthesis of these two elements. But this crux is also a failure: as we show in this section, existential risk analysis is constituted by a radical mismatch between method and object of study. This chapter also examines more closely the claim that existential risk work is scientific and not science fiction, which ignores the theoretical and predictive insights available in the imaginative work of literary speculation.

Chapter 3 provides a more expanded history and definition of existential thought. We claim that many philosophers associated with existentialism, and many subsequent philosophers critical of these initial existential positions, provide a compelling reflection on why existential thought and existential risk must be coupled together. We also discuss why not one single philosopher in the field of existential risk has put any effort into reflecting on the history and insights of what “existential” means. This chapter examines more closely three rejections of Bostrom-style existential risk philosophizing, first in the work of contemporary theorists Claire Colebrook and Ray Brassier, and then in the historical contributions of Hans Jonas. The chapter closes with a consideration of what kind of scale critique is most effective for existential ecological thought, turning to questions of cosmology and eschatology that are often cryptically at stake in studies of existential risk. Finally, the book’s conclusion examines the rhetorical lures of Bostrom’s utopian writings, specifically his “Letter from Utopia.”

We base our arguments in an ecological–existential framework, but this is not a claim that we must forever leave nature alone; Bostrom is right to be skeptical that we must defer to “the natural order” as the sole standard of the good.27 We do not view ecology and the existential condition as unchanging origins and fixed ends. Ecological existence is dynamic, made and mediated by biological relations, cultural practices, technological changes, and imaginative pursuits. We are already cyborgs, but what kinds and with what futures? We support technological improvements to alleviate suffering and improve the function of ecosystems. Yet we do not support the idea that all of the planet and all of the existential condition should be under technological control and intelligent enhancement in order to mitigate and master existential risks. We are against species-wide irreversible technological changes, changes that would be made without consultation or consent. We are against changing the demos (the lived world) by nondemocratic means. The planet already is a calamity for many species in recent centuries, and many human communities exist on a daily basis in existentially oppressive conditions. We want a better world through technology, but not through technology alone, and not requiring a dependence on the exploitation, extractivism, inequality, and massive economic disparity that facilitates current hyperindustrial capitalism. And we reject Bostrom’s claim that transhuman enhancements will become the new categorical imperative as long as they pose no existential risks. While there is no metaphysical necessity to be drawn from nature, there is also no metaphysical necessity for transhumanism.

These three critiques are not primarily moral criticisms. Rather they are oriented around methodological and ontological disputes with existential risk theory, although our points do have moral consequences that we defend as implicated in our method. To put our cards on the table, we envision the long-term tasks of humanity as follows: share and tend the Earth, work toward symbiosis in the sense of evolutionary mutualism and social collectivity, harness the energy of the sun, and communicate with the galaxy. To advance this vision, we will need new forms of address and belonging, as well as new forms of accountability and collaborative exploratory epistemologies, on a planetary scale. We will need solar energy technologies and new solar cultures that cultivate solar commons and collaborations rather than solar sovereigns and inequity exacerbated by solar capitalism for the few. This vision informs our methodological assessment of the long-term horizons of existential risk and our own understanding of the existential condition.

This book, however, is unapologetically on the whole a work of negative critique. Much of the work of theory remains unmasking, demystification, and showing the unspoken historical and philosophical conditions of discourses such as existential risk—and this work is inclusive of more affirmative and creative conceptualizations. If existential risk were only an obscure philosophical subfield, it might not warrant this treatment. Why not reason with “no-nonsense” straight talk about the mitigation of possible human extinction, using whatever methods seem right to intellectuals who want to go there? Since existential risk has risen to a fairly high level of popularity and media impact, the field deserves close scrutiny from critics with a background in the study of related topics across the sciences and humanities. We have not been the only ones, and we hope others will continue this effort in the future. We also acknowledge that negative critique has obvious limitations in terms of providing immediate relief or resolution for existential risks. While that is not our primary purpose here, we note that critique, as well as fostering existential commons and reasoning through practices of care, relation building, economic equity, and environmental attentiveness, model ways to share a world—thus mitigating existential risk. A combination of patience, critique, sociability, imagination, and scientific reason may take us further in crafting just and habitable worlds than wide-eyed apocalyptic doomscrolling or the fantasy of living “long enough to live forever,” as Ray Kurzweil puts it.

Finally, there is a cosmological undercurrent in this book that overlaps with its interest in risk, apocalypticism, utopianism, and most of all, existential ecology. If there is one thing that mitigates our antagonism toward existential risk (beyond the obvious need to raise awareness about future horrors), it is this new excitement about cosmology. We don’t know where it will go, but we have begun to wonder why there is so little interest in the topic in Left critical humanities circles. After all, one way of explaining existential risk’s incongruous mixture of sci-fi fantasy plots and dry calculation is that it taps a deep vein of public desire for secular eschatology—that is, for stories and scenarios about the origins and ends of humanity, life, and the universe. Inquiries on ontological domains at vast scales and strange thresholds are not separate from attention to environmental and social justice on Earth, which include care for existential pluralism and respect for cross-cultural attunements to cosmological conjectures. Such a perspective requires cultivating what Marisol de la Cadena and Mario Blaser call a “pluriverse” predicated on “the practice of a world of many worlds, . . . heterogeneous worldings coming together as a political ecology of practices, negotiating their difficult being together in heterogeneity.”28 While cosmological concepts intertwined with existential risk are not this book’s main strand, we have been glad to follow them in unexpected directions.