13. Wicked Systems and the Friction between “Old Theory and New Data” in Biology, Social Science, and Archaeology

Claes Andersson, Anton Törnberg, and Petter Törnberg

Some of the most challenging and important problems facing us both scientifically and as citizens emanate from large-scale complex adaptive systems, such as societies and ecosystems. Diverse examples include social exclusion, credit crises, environmental unsustainability, biological evolution (with its myriad subproblems), and the evolution of hominin culture. These problems are not only important and complex. They are also interlinked in bewildering ways, are difficult to even define or delimit, and appear to have become increasingly pressing lately. We seem to see and feel the shortcomings of our understanding more acutely now than we used to only a few decades ago.

The reasons for the increasing saliency of these sorts of problems are manifold, and they are expressed differently in different fields, but we believe that three important and interlinked factors can be identified: (1) theories that used to be thought of as safe ground are being undermined by (2) an explosion of new data, while at the same time, (3) the development of complexity science has provided models and concepts for expressing and detecting these types of problems. For societal problems, an additional point is germane: the societal system itself is becoming more and more complex (i.e., more interconnected, less predictable, and less stable, as well as more and more energy- and material-intensive, with a stronger effect on ecology and climate as a result).

We believe a major transformation is occurring that spans several disciplines. This transformation is unfolding at different paces and along somewhat different trajectories in different disciplines, which reflects differences in theoretical and empirical backgrounds, in what constitutes central questions and aims. It is generating a growing substrate of semicongruent critiques and new ideas, but an understanding of what is wrong with “old theory” is, overall, more developed than an account of what would work better. The situation presents a need, and indeed an opportunity, for conceptual tools that act to align and direct this substrate of critiques and new ideas on an abstract level so that it can reach across disciplinary boundaries.

This chapter aims to contribute some elements of such a toolbox. We begin with a review and analysis of recent empirical and theoretical trajectories, in and across three important areas where this transformation is having strong effects: evolutionary biology, social science, and archaeology. Our exploration leads to the introduction of two interrelated tools: a metarepresentational diagram and a new class of wicked systems. We probe the value of these tools by addressing questions such as: What is “wickedness”? What approaches have been applied successfully to understanding it? What can we do to make further headway?

Transformations in and across Disciplines

Evolutionary biology is undergoing a dramatic transformation driven by strong empirical advances that have occurred over the past two decades. These advances have been interpreted as revealing a mechanistic basis for evolution (Wagner, Chiu, and Laubichler 2000; Laubichler and Maienschein 2013; Laubichler and Renn 2015)—from molecular to ecological scales—at a level of detail that was hardly imaginable only twenty years ago. In this emerging picture of evolution, age-old disciplinary boundaries break down, and trusted models are being undermined, both in terms of the predictions they make and the assumptions that underpin them (e.g., Erwin 2008; Odling-Smee et al. 2013; Laland 2014). Yet this new picture is not yet unified. It emerges—tantalizing in outline but still somewhat out of focus—from a range of perspectives on the evolutionary process, including the complex mapping between genotype and phenotype and the structuring of phenotypic spaces, the role of organization and history in evolution, multiple channels of inheritance, selection on multiple levels, and macroevolutionary patterns.

These new perspectives are embodied as an evolving system of diverse and interacting theoretical elements, such as evolutionary developmental biology (e.g., Arthur 2011), niche construction theory and ecological inheritance (e.g., Odling-Smee, Laland, and Feldman 2003), ecosystems engineering (e.g., Jones, Lawton, and Shachak 1996), ecoevolutionary dynamics (e.g., Pelletier, Garant, and Hendry 2009; Loreau 2010), facilitated variation theory (e.g., Bruno, Stachowicz, and Bertness 2003; Gerhart and Kirschner 2007), developmental systems theory (e.g., Oyama, Griffiths, and Gray 2001), generative entrenchment (e.g., Wimsatt 1986, 2001), developmental innovation (e.g., Erwin and Krakauer 2004), and “public goods” theories of evolutionary transitions (e.g., Erwin and Valentine 2013; Erwin 2015). The basis for this new system of theories derives from empirical fields enabled by technological advances, such as comparative genomics and developmental genetics (e.g., O’Brien et al., 1999). The lesson is that biological evolution is a much broader and more complex problem than we previously imagined.

Laubichler and Maienschein (2013) identify two alternative narratives about the history and future of evolutionary theory. The first, and most widespread, is that we are seeing a completion of the Modern Synthesis—that is, an “extended synthesis” (e.g., Pigliucci and Müller 2010) where this new mechanistic understanding is being accommodated in a cumulative fashion and within current theoretical frameworks. The second is that there is a more fundamental challenge to how evolutionary dynamics is understood in the Modern Synthesis. Laubichler and Maienschein (2013) argue that we are seeing the emergence of a new “causal mechanistic” evolutionary biology that has its roots in old complementary approaches to evolutionary biology (Laubichler and Maienschein 2007).

As the detailed mechanistic basis of biological evolution is comprehended better and better, evolutionary biology is increasingly being forced to view its subject area similar to the way that qualitative social sciences and the humanities have always viewed their subject areas. In this view, biological evolution is composed of hierarchical and historical complex systems in which contingent details matter greatly, problems and subsystems are potentially impossible to delimit, and important interconnections exist across levels of organization. This recent confluence between biology and social science appears to be “spontaneous”—that is, there is no suggestion that theoretical exchanges across this academic divide drove these developments. The emerging view of biology reveals features that are deeply congruent with corresponding features of societal systems and, as a consequence, has directed theory, problem formulation, and debates in similar directions.

This new way of viewing biological evolution enlists historical and interpretative approaches, akin to the narrative case studies used in qualitative social science (e.g., Ragin 2009; Byrne and Callaghan 2014), and is evident in the novel theoretical trajectories listed above. Specific, contingent processes and histories frequently must be described with diagrams and narratives in order to capture their causal structure. It is not a matter of abandoning formal modeling; rather, it is the limitations of various formalisms that have become more acutely felt. The emerging empirical picture of evolution forces biologists to go outside the bounds that formal modeling imposes on inquiry.

The social sciences have a long tradition of qualitative theorizing about the detailed causal structure of society. This has nurtured an internal animosity and fragmentation between qualitative and formal quantitative approaches. The social sciences are also under increasing pressure to deliver in terms of policy. This pressure emanates from the empirical developments of information and communication technology, as well as from new demands in a rapidly changing reality (e.g., Beddoe et al. 2009; Zalasiewicz et al. 2011; Steffen et al. 2015). Not least, a mounting scale and frequency of societal and environmental crises—a “metacrisis” (Lane and van der Leeuw 2011)—has made the limits of our understanding and our control over society and the global environment simultaneously more obvious and threatening.

In the wake of this metacrisis, which was neither predicted nor hindered by our current understanding, there is a widespread sentiment that we must broaden the range of factors we think affect the direction of society: from the primacy of economic values to an inclusion of societal and environmental values; from a reductionist view to a more holistic and inclusive view. Although there is no consensus about what this really means or entails, most now agree that society is highly complex, and we must attend to its complexity much more explicitly.

All of these developments have changed the landscape for policy, which represents a normative dimension that biology and archaeology largely lack (see, e.g., Byrne 2005; Scoones et al. 2007; Leach, Scoones, and Stirling 2010; also, reflecting this fragmentation, Ball 2012; Helbing 2013). The question of how to predict and optimize the future is yielding to an acceptance of the futility of such aims and an embrace of other goals, such as resilience and sustainability. On the one hand, this raises serious questions about the efficacy of many standard policy tools, most of which were designed under different assumptions about how societal systems work (most notably, neoclassical economics). Indeed, this challenges even our basic intuitions about how societies evolve. But, on the other hand, it also has opened up the promise of entirely new types of analytical tools, based on ideas about how we can dynamically steer and scaffold society by engaging more directly with its causal mechanics. Possibilities include more bottom-up approaches like the management and design of social networks of actors (e.g., Lane and van der Leeuw 2011), the historical study of sociotechnical transitions (e.g., Geels 2002), or the management of innovation “pathways” (e.g., Leach, Scoones, and Stirling 2007, 2010; Loorbach 2010; Wise et al. 2014).

Archaeology and paleoanthropology provide an interesting third case, not least because biological and societal evolution fuse in a common coevolutionary history. These fields have been hit by an empirical revolution that is at least as dramatic as the one described for evolutionary biology. Again, new and improved laboratory techniques constituted a major driving force; for example, in biomolecular analysis (e.g., Brown and Brown 2013), use-wear analysis (e.g., Lerner et al. 2007), palynology (e.g., Holt and Bennett 2014), dating techniques (e.g., Aitken 2014), and methodology (Tostevin 2012). Details about the lives of ancient hominins that were unimaginable until lately are now coming to light (e.g., Kristiansen 2014); the mechanistic basis of cultural evolution is being revealed, and similar to biology, there is considerable friction between “old theory and new data.”

Two dominant “old theory” approaches to understanding Paleolithic cultural evolution can be characterized in a schematic fashion:

1. 1. A cognitive/physiological approach (CPA) that emphasizes cognition, as both an enabler of and constraint on culture. The CPA is pervasive but is rarely championed explicitly; Richard Klein typically serves as its embodiment (and lightning rod) in the literature (e.g., Klein and Edgar 2002). Its logic dictates that periods of cultural stability must express what is maximally attainable at certain levels of cognitive capability because strong selection would rapidly exhaust cognitive potential to produce adaptive artifacts and strategies. Transitions, consequently, would be the result of genetic novelty that confer new and distinct behavioral “packages.”
2. 2. An ecological/economical approach (EEA) largely based on behavioral ecology and economic constraints like time consumption and energy costs (inter alia). The EEA is often contrasted to the CPA approach, which focuses more on artifacts and hominin taxonomy. The EEA emphasizes geographically and temporally varying environmental selection pressures as the prime mover of change in the past (e.g., Foley and Gamble 2009). In this approach, cognition is seen more as a contributor to variability than as a set of fixed capacities.

Neither of these approaches is sufficient on its own, and controversy typically concerns the relative importance of the features they emphasize. Moreover, both share a common Modern Synthesis model of adaptation where the fitness of physiological and cultural expressions, relative to an external environment, is the sole provider of evolutionary direction and where evolution is a process of constrained optimization.

As we move into the Holocene with sedentary farming communities, the discourse shifts weight from biology to sociology and anthropology; the EEA remains, but the CPA falls to the side as cognitive evolution loses its centrality.

This is natural since the more recent empirical record presents archaeologists with considerably more detail than earlier periods. But, despite this different theoretical emphasis, the prescriptions of older universal models in those traditions are not that different from those of the CPA and neither is the friction caused by emerging empirical patterns. For example, the idea of transition as a sudden appearance of a new “package” has been equally important in Neolithic research (e.g., Çilingiroglu 2005; Barker 2006) as it has been for Paleolithic research (e.g., McBrearty and Brooks 2000; Belfer-Cohen and Hovers 2010).

Formal Darwinian approaches to cultural evolution were introduced and developed starting in the early 1980s (e.g., dual-inheritance theory and evolutionary archaeology; see Cavalli-Sforza and Feldman 1981; Boyd and Richerson 1985; Mesoudi 2011). These go outside of the mainstream that the CPA and EEA describe, but they largely fit into the same pattern as they seek to identify a unified theoretical basis from a fundamental principle of organization (e.g., population thinking). These approaches adapt models from population genetics and rational choice theory, with population dynamics coming out as so fundamental and dominant that other factors become secondary or peripheral.

The new wealth of detailed data is being interpreted in terms of a more gradualist pattern (e.g., McBrearty and Brooks 2000; McBrearty 2007; Maher, Richter, and Stock 2012), often characterized as more complex, messy, intermittent, and in need of attention to detail (e.g., Barker 2006; Hovers and Kuhn 2006; Habgood and Franklin 2008; Belfer-Cohen and Goring-Morris 2011; Hovers and Belfer-Cohen 2013). There is a search for new theoretical traction (e.g., Hauser 2012; Zeder and Smith 2009; Zeder 2014; Stiner et al. 2014; Stiner and Kuhn 2016) as older prime mover and single origins theories are undermined.

Overall, the weight of evidence tells us that ecological, evolutionary, and societal systems do not work as previously assumed for the sake of methodological expediency. In and across these disciplinary fields, new empirical knowledge undermines old theory in three major ways: (1) it buttresses old complaints about poor predictions, explanations, and policy advice from these traditional approaches, (2) it refutes central assumptions, many with an axiomatic status, that underpin old theory, and (3) old theory is frequently an obstacle to making sense of these new data. The reason is that we are not just dealing with more data but new types of data, which older theory was designed specifically to ignore since they could not be accessed with confidence. As a consequence, old theory is frequently criticized for not being extendable in the required directions, necessitating more radical theoretical innovation.

A wealth of new theoretical elements emerges in this friction between old theory and new data, but they are at present not strongly aligned, not even within the fields. The “search for new theoretical traction” is still very much unfolding; obtaining it is the challenge and opportunity that lies before us.

A Wicked Theoretical Crisis

What we see in this emerging picture is the outline of a class of systems that exhibit a deep similarity and encompass both societies and ecosystems. This similarity provides a common platform from which to search for new theory. The outline revolves around features known to be methodologically problematic, the key elements of this new empirical picture (e.g., complexity, lacking clear levels of organization, heterogeneous structure, etc.). The proposed deep similarity is expressed as similar sets of problems, theoretical responses, debates, models, and concepts; it crystallizes more and more clearly as we explore and discover more about these systems. Call this class of systems wicked systems because they are distinct from and yet related to complex systems, which would otherwise be a natural label.

The label wicked systems accents a potentially deep connection (whose exact nature remains to be worked out) between this class of systems and what have been called wicked problems in social science. The term wicked problems was first coined in management research by Horst Rittel (briefly introduced by West Churchman [1967]) to characterize a class of problems that did not fit into the mold of formal systems theoretical models that were being applied widely and with considerable confidence at the time. Most large-scale societal problems fall naturally into the category of wicked problems: starvation, climate change, geopolitical conflicts, social disenfranchisement, and so on. These problems resist definitional characterization, and the efficacy of proposed solutions is called into question frequently, not only with regard to feasibility and adequacy but also with respect to the risk of creating cascades of unforeseen problems that may be worse than the initial problem (see also Leach, Scoones, and Stirling 2007; Scoones et al. 2007). With wicked problems, we either tame them by creating “an aura of good feeling and consensus” or by “carving off a piece of the problem and finding a rational and feasible solution to this piece” (West Churchman 1967). This also describes the problems we discussed initially across all three fields, encapsulating the troubles we face when applying old theory to new data. By considering “wickedness” as a system quality, we can generalize to speak of wicked dynamics, wicked phenomena, and wicked systems. This allows us to refer to these crises and transformations in a unified way and articulate the growing realization that the sciences must face wickedness more directly on its own terms. But how? Will our old weapons and battle plans work? Can they be incrementally changed and combined to meet the challenge? What sort of understanding, prediction, or control can we expect, realistically? To begin answering such questions, we need to better characterize wickedness as a system quality. One place to start is with a review of approaches that have been applied historically to evaluate how they have succeeded and failed.

How Do We Deal with Wickedness?

A battery of approaches have been used in the past to deal with wicked systems. These approaches fall into four broad categories: narrative theory, analytical models, systems theory, and complexity science. Narrative-based theory—basically disciplined or systematic thinking and communication—is very old, whereas the latter three formal approaches are newer additions to the toolbox that we apply to understand the world (Figure 13.1). The confidence we place in these to understand wicked systems is typically buttressed by a strong track record of success in understanding other, less unruly, systems (often in the physical sciences). They encourage us to see that we only need bring these unrulier systems under the umbrella of “proper science” and away from the interminable talk, hairsplitting, and subjective opinion that is seen as inherent to narrative approaches. But we are still waiting for the breakthrough, and, to varying degrees, there are signs of stagnation in all three approaches.

Where does this leave us with regard to wicked systems? Although complexity is a crucial concept and complexity science arouses the most enthusiasm as a problem-solving strategy, the latter appears to have hit considerable resistance in the face of these systems. Although concepts like path-dependency, attractors, tipping points, and chaos have transformed ideas about causality in society and biology, these highly general lessons have proven hard to operationalize for wicked systems. Complexity science appears to offer a perpetual promissory note, but by examining its lack of achievements, we can glean insights into how these systems work.

Complexity, as invoked so far, is not well defined; that is precisely our point of entry for the remainder of our exploration. Setting aside the vast and fragmented literature that attempts to define complexity, both because it is unnecessary and likely a misguided project, we will approach complexity ostensively by assuming that “complexity is what complexity science does (well).”

Complex, Complicated, and Wicked

Complexity scientists often distinguish between complexity and complicatedness (or dynamical versus structural complexity; see, e.g., Erdi 2008). These two system qualities are often contrasted for the purpose of explaining what complexity science focuses on: complexity is associated with bottom-up self-organization, such as the behavior of a school of fish or a crowd, whereas complicatedness is associated with top-down organization, such as in engineering. Though not a formal definition, it helps to illuminate the practice of complexity science, which deals with complexity, not complicatedness, even though the latter can be seen as a subset of the former.

The history of complexity science helps to illuminate how this practice and (largely tacit) meaning of the term complexity emerged. The Santa Fe Institute (SFI) acted as a powerful uniting and aligning force in what today is referred to as complexity science. Founded in 1984 by a group of highly influential scientists, many of whom were active at the nearby Los Alamos National Laboratory, the SFI was the first dedicated research center for complexity science. Because of the founders, it was tightly linked to the origins of scientific computing and dynamic systems theory (see, e.g., Galison 1997). Although many important ideas about complexity predate SFI, such as are found in qualitative social science and systems theory (see, e.g., Sawyer 2005; Vasileiadou and Safarzyska 2010), it remains the case that the SFI came to define a mainstream of complexity science and thereby also, in practice, the concept of complexity as understood by scientists, policymakers, and the public.

The SFI was created as a multidisciplinary center. Although it remains highly multidisciplinary, it is not as methodologically diversified. The primary methodology that was (and still is) pursued at the SFI is formal and quantitative, much closer to natural science and quantitative social science.1 Computer simulation is at the heart of this methodology, which puts into motion the entities and interaction rules of dynamic systems. This extremely flexible methodology makes it possible to study and visualize dynamics that were previously inaccessible to the human mind—aided or unaided. Above all, it makes possible a systematic inquiry into emergent properties in dynamic systems. This capability provided a powerful impetus to the formation of complexity science worldwide with the SFI as its central hub.

The typical model in this tradition has a microlevel of abstract agents or nodes existing in a predefined environment. Complexity scientists study and probe the patterns that arise on an emergent macrolevel from the dynamic interaction between these agents or nodes. This is what complexity science does well. Individual traditions and scientists may be more or less strongly aligned with it, but anyone claiming to work with “complex systems” must relate to this methodology in one way or another. Thus, complexity is a concept whose meaning is constructed mainly by the complexity science community working with it.

Making this typical construal of complexity explicit helps to reveal the limitations of its applicability and delimit the class of systems that are amenable to analysis using it.2 In short, although complexity and complicatedness are linked in numerous ways, they present radically different sets of methodological and theoretical challenges.

Are societal systems and ecosystems complex or complicated? On the one hand, they are undeniably complicated, with multilevel organization and a bewildering array of qualitatively different and interacting entities. Systems theories seize upon what appears to be an irreducible complicatedness of societal systems. Yet society is also a complex system in the bottom-up self-organization sense (e.g., Sawyer 2005; Castellani and Hafferty 2009; Ball 2012). One can even argue that much of its complicated structure arises from bottom-up rather than top-down processes. The story is similar for ecosystems. There is no reason why systems cannot be both complicated and complex at the same time; our two wicked systems appear to be excellent examples of this type of system.

Figure 13.2 uses these two dimensions as axes to map out a space of possibilities (see Andersson, Törnberg, and Törnberg [2014b] for a discussion focused on societal systems). We thereby obtain a separation between types of systems that we otherwise tend to conflate as “highly complex” and our wicked systems cluster where both dimensions are emphasized (i.e., complex and complicated). Surprisingly, the possibility of systematically exploring the consequences of systems exhibiting both complexity and complicatedness has not been pursued explicitly.3 Complexity science may be aware that complexity and complicatedness are distinct qualities, but complicatedness in complex systems is not seen as a fundamental problem. That they are complex is fundamentally important; extending mainstream complexity science to deal with them is seen as challenging but, essentially, gradual and cumulative work. However, wicked systems are not a type of complex system but rather fall within a system where complexity and complicatedness are both present. This combination—wickedness—is not something that complexity science, systems approaches, analytical models, or combinations thereof address very well.

Understanding the Coevolution of Methods, Problems, and Systems

Why is it so difficult to extend mainstream complexity science to wicked systems? This is a question about the relations between methods, problems, and systems. The answer boils down to why formal approaches, in general, will be incapable of dealing comprehensively with these systems. In Figure 13.3, the four basic approaches—narrative theory, analytical models, systems theory, and complexity science—are mapped onto the complexity–complicatedness plane. Narrative approaches are not married to assumptions of low complexity and complicatedness, but they quickly run into problems when complexity or complicatedness become too prevalent. Thus, narrative approaches fall roughly in the middle of the diagram. The introduction of analytical models over the past few centuries, such as Newtonian physics, neoclassical economics, and Modern Synthesis evolutionary biology, achieve analytical power by abstracting away from the richness of real-world systems (much of which was inaccessible empirically). These successes, especially in the natural sciences, dovetail with the Platonic idea that nature is at the bottom, governed by simple and elegant laws that the sciences uncover. In the mid-twentieth century, the “complicated flank” was occupied increasingly with the development of general systems theory and cybernetics. This was motivated in part by a need to match the macroscopic organization of systems but also by the conviction that elegant laws resided in holistic systems. As cheap computing became widely available in the 1980s, complexity science entered the scene and covered the “complex flank.” The search for universal laws focused on emergent patterns in dynamic systems.

Formal approaches are unable to address many of the problems that wicked systems present us with (see above, section 2). Instead, they selectively address subproblems that happen to fall in their domains or transplant problems from near the wicked corner to the corners of their methodological preference (Figure 13.4). In the former case, important but limited “snapshots” of the system in question can be obtained, typically with a taste of revealing laws of great generality. Although these often reveal important major principles, we are faced with the problem of how to combine the snapshots (see also Wimsatt 1975). In the latter case, we may get spurious results because strong assumptions mean that the benefit of using formal methods of analysis does not warrant the price in realism.

What Figures 13.2 and 13.3 primarily accent is that there is a theoretical lacuna for wicked systems. This theoretical lacuna does not emerge clearly unless we systematically make a separation between complexity and complicatedness. But our diagram also emphasizes further questions: What is “wickedness”? Why is it so analytically recalcitrant?

The Genesis of Wicked Systems

Complexity and complicatedness can be seen as mutually reinforcing in our two principal examples of wicked systems: societies and ecosystems. Self-organization generates, changes, and maintains macrostructure. Macrostructure, in turn, scaffolds and creates a multitude of arenas for self-organization. How do wicked systems originate? How do complexity and complicatedness become fused into wickedness? How are wicked systems maintained? What sets them apart from systems where either complexity or complicatedness dominates?

Complicated systems, such as machines and organisms, have distinct life cycles and tend to be adapted to specific functions in the context of an external environment. They have an initial phase of assembly or development, during which they are shielded from the rigors that face the completed system during a subsequent use phase. Automobiles, for example, get assembled according to strict plans, and they are used as automobiles only once fully assembled. Organisms develop under some use-phase requirements, the most basic being they must remain alive throughout development, and juveniles of many species (not least, humans) gradually become exposed to the rigors of adult life, while metamorphosing species undergo one or more transitions between different ecologically adapted forms. What we describe is more salient in K-strategists; r-strategists invest in large numbers rather than robustness in, and shielding of, the offspring. In both cases, investments are needed to minimize the effects of conflicts between functional and developmental requirements. The life spans of complicated systems are usually sufficiently short, so the environments they are adapted to can be assumed to change very little. They are fundamentally not designed to be very flexibility and their flexibility mainly resides between rather than within life cycles, such as in design processes, variation and selection regimes, and so on. The strong and fundamental nature of this constraint on evolutionary adaptation is evident from the high investment that organs provide even limited intragenerational adaptability represent—for example, the adaptive immune system and brains that provide a capacity for learning or even culture.

Wicked systems are complicated, but they are neither designed nor assembled. They have contingent histories and lack a separation between assembly and use phases. However, they also are complex, though they do not self-assemble from simple (e.g., regular or random) initial states, which is how complexity science models complex systems. Change, adaptation, and mitigation in these systems typically constitute what is meant by wicked problems (see above, section 6).

Something that seems to characterize adapted complex and complicated systems is the lack of incentive for parts to benefit at the expense of the whole. Along with external shielding during an assembly phase, this is an important factor that lends flexibility and precision to their processes of formation: they may be designed, assembled, and changed “in peace” so as to fulfill an overall functionality. When interactions are not necessarily symbiotic, such as when humans are part of social systems, one finds distinct mechanisms for eliminating such interactions and for enforcing an overall alignment of the parts (e.g., various forms of detection and policing, such as by immune systems, legal systems, intelligence agencies, the military, etc.). The relation between somatic and germ line cells (e.g., Michod and Nedelcu 2003) represents the most potent example because it signifies how even the potential of parts to rebel against the whole is eliminated.

Wicked systems are open arenas for complex interactions between complicated systems, where these interact freely (complexity), creating and dissolving structure (complicatedness) across levels of organization and constantly changing the environment for the interacting components (again, complexity). In these arenas, we find a range of types of interactions. Sandén and Hillman (2011) describe this heterogeneity of interactions in society using terminology borrowed from ecology. These stem from the set of possible outcomes for agents in pair-wise interaction: favorable (+), neutral (0), and unfavorable (−). For three or more agents, there are symbiotic (++), competitive (−), neutral (00), parasitic (+−), commensal (+0), and amensal (−0) interactions. The presence of a full set of ecological interactions is central to the quality of wickedness because the organization of such systems is constantly in upheaval—it never settles into something that is easy to understand, adapt, or design.

Wicked Systems as Poorly Decomposable Systems

It is not hard to imagine why systems exhibiting this range of interactions would be challenging to understand formally, but can we understand formally why they cannot be understood formally? This could offer a more detailed map of the exact methodological problems with these systems. Such an understanding might inform us about what approaches are likely to work in particular contexts and could serve to make it easier for formalists and nonformalists to collaborate. One way of understanding why wicked systems are so recalcitrant that is both formal and intuitive and accessible to formalists and nonformalists alike is in terms of not exhibiting near-decomposability (Simon 1996). Near-decomposability turns out to be necessary for almost all formal theorizing.

Simon (1996) introduced the concept of near-decomposability to explain in a clear and systematic way what conditions need to be fulfilled for a system to be studied in a formal and controlled manner. In order to study a system in isolation, its dynamics cannot be disturbed significantly by outside influences. We should be able to identify an internal environment where the dynamics under scrutiny take place and an external environment that is assumed to be stable, or variable only in highly regular ways. The boundary between the internal and external environment is referred to as the interface (Figure 13.5). What we study with a model is the internal environment. Hierarchical system organization is also important: our internal environment constitutes the external environment of the objects that populate it. Objects are dealt with only in the form of interfaces, such as via their interactions with other objects in the studied system. The beauty of all of this is that it makes the world manageable: we declare our system autonomous from external disturbance and hide any complexity or complicatedness at lower levels of the hierarchical organization.

We study this internal environment on a timescale that is long enough for our objects’ interfaces to be meaningful and for important dynamics to have time to occur and short enough for our assumptions about the interfaces to remain valid.4 The greater the separation of scales between the internal and the external environment, the greater the difference in size and speed of the dynamics on these two levels, giving more interesting things time to happen. For example, models of particle physics can be formulated in this way because those systems exhibit clear scale separation. Engineered systems are designed to fit this description (Simon 1996; see Figure 13.6).

In many important cases, we can make assumptions of near-decomposability for wicked systems and thereby bring powerful scientific approaches to bear on them. Certain subsystems, such as crowd behavior, protein folding, or the ceteris paribus fate of a new trait in a population, can fit this description and become amenable to complexity science modeling. The dynamics of cars and people play out over much shorter timescales than urban systems, roads, and traffic regulation change. Such phenomena are often ephemeral, which bounds the problem even further. For example, at night the traffic jam dissipates and leaves no traces that affect tomorrow’s traffic. Similar features obtain for abstractly conceived phenomena that depend on persistent features, such as network dynamics, geography, basic resource constraints, or strategic dilemmas.

What about evolutionary and ecological phenomena more generally? For example, what about sociotechnical transitions, evolutionary radiation events, or other wicked problems? Wicked systems are open systems in which many and diverse types of processes coexist, coevolve, and have an impact on each other across overlapping timescales and levels of organization. They involve discontinuous, qualitative change as well as cascade effects (e.g., Lane 2011), whereby change strongly and rapidly feeds back on the conditions for further change. Such systems are difficult to contain in a suitable timescale for transitions to be studied against the background of an unchanging external environment. The fundamental problem in this context for complexity science, or any approach that relies on these ontological assumptions, is that there is no way of partitioning wicked systems into distinct and persistent levels of organization (Figure 13.7).

We began by reviewing recent developments in evolutionary biology, archaeology, and social science, pointing out interconnections and similarities in terms of their problems, debates, new ideas, and new data. A historical dearth of data—both in terms of volume and types of data—made it easier to base elegant theoretical models upon strong assumptions about complexities that were unknown. Considerations of methodological expediency were thereby not resisted by empirical knowledge about the causal and material bases. Strong general bodies of theory, emphasizing the broad strokes, emerged in this environment, as did disciplines dedicated to their study. But merit is not the only thing that conserves these theories and disciplines and the boundaries that we have erected between them: they have been subject to generative entrenchment and hardening. This is what we refer to as old theory. New data is much more multifaceted, and voluminous bodies of empirical knowledge that call into question many of those generatively entrenched basic simplifying assumptions—about human behavior, culture, development, and so on—are now causing friction against the hardened pillars of old theory.

The cautions that we would like to make are, first, that the conclusion that old theory can accommodate old data is a convenient one and should be scrutinized critically: it may be so, but there are structural reasons why we may shun the effort of going for more fundamental reevaluations. Another reason is that generative entrenchment partly consists in, precisely, a priming of our imagination as we innovatively search for new solutions: it has accustomed us to think about dynamics and causation, what legitimate problems look like, what tools are deemed secure, and so on. This type of constraining facilitation of innovation has many names across these fields, such as facilitated variation in evo-devo (Gerhart and Kirschner 2007), design spaces in innovation theory (Stankiewicz 2000), and developmental canalization in behavioral biology. The second caution is that we live in an “innovation society,” in times that value novelty and change highly (Lane 2016). There is the opposite risk of overreaction and of equating “old” with “bad.” What is called for, we believe, is a delicate and thoughtful reevaluation of the old in the light of the new.

The emerging empirical pictures in these fields are frequently referred to as a complex. To get a better idea about these issues we factored complexity into two component qualities: complexity and complicatedness—corresponding to dynamic and structural complexity, respectively. This made it possible to map four different approaches (narrative theory, analytical models, systems theory, and complexity science) onto distinct system classes that are simple, complex but not complicated, complicated but not complex, or both complex and complicated (i.e., wicked). This mapping displayed interactions between methods, systems, and problems, while leaving a conspicuous lacuna near the “wicked corner.”

We argued that the described friction may, moreover, be understood as a much broader encounter with a distinct system quality: wickedness. The term originated, indeed, precisely in the interaction between strictly formulated systems theory and the societal problems to which they were applied (Rittel and Webber 1973); the friction in this case did not have to wait for advanced empirical instruments to emerge.

Next, we turned to answering how wicked systems work and where they come from, aiming to differentiate them from, yet keeping them in relation to, complex and complicated systems. Wicked systems were deemed best described as arenas for complex interaction between complicated systems. They remain “enclosed” and in recurrent interaction, but the absence of effective top-down alignment means that any structure that emerges will constantly be in flux, with constant qualitative change flowing up and down a poorly scale-separated hierarchy. As a consequence, they conform poorly to the key requirements needed for formal modeling to be effective, such as near-decomposability (Simon 1996).

Even though wicked systems are distinct from complex and complicated systems, they are typically addressed as if they belonged to one of these classes. Grappling with wicked systems as distinct types of systems is a promising direction for future research. Any resulting understanding could contribute to a more systematic metatheoretical discussion, foster new transdisciplinary connections, and improve our capability to use new empirical assets and tackle pressing problems.

Notes

References