Methodologies: Concept Work and Fieldwork in Philosophy

Rabinow describes concept work as a methodological orientation originating from the general collapsing together of the human disciplines, and anthropology and sociology specifically. Concept work is a methodology continuing social scientific work amid the “ruins” of various humanist approaches.7 Concept work involves experimentation with different analytic frames (e.g., “theoretical frameworks”) rather than simply using a single frame or even an assembled frame to posit or represent a problem and its potential solutions as a singular entity. Thus, concept work is designed to problematize situations rather than generate answers or solutions to pregiven problems. In a similar vein, Colebrook and Weinstein contend, “What is required [today] are not answers to questions but a reconfiguration of the forces from which questions emerge.”8 In short, concept work is an approach that attempts to rearrange situations, problems, and diagnostic frames in a concerted effort to posit “novel problems, not answers.”9

Our use of concept work is based around trying to understand emerging forms of education governance, or how education governance adapts, evolves, grows, mutates, and ostensibly widens its own limits within the overabundance of data available to it. We aim to reconfigure our empirical work in a philosophical register, but more importantly, to develop a register that entertains multiple ontological and epistemological confluences, in contrast to simply presenting our findings as singular statements about governance, power, and force. We see concept work as congruent with the philosophical activity that Deleuze and Guattari undertook when they defined philosophy as “forming, inventing, and fabricating concepts.”10 What Bourdieu characterizes as a “fieldwork in philosophy” also inspires our approach. Bourdieu’s sociological method seeks to explore the possibilities of “a combination of concrete empirical analyses and practical philosophical considerations.”11 Heimans proposes that fieldwork in philosophy “is formed out of an ongoing unseparated engagement between fieldwork and philosophy.”12 Concept work and fieldwork in philosophy are both analytical and synthetic, with the empirical serving as a basis for theoretical development.

Mol terms this kind of methodological approach “empirical philosophy.” The major shift with empirical philosophy (i.e., concept work and fieldwork in philosophy) is away from traditional concerns about epistemology and “knowledge” that stand apart from practices, and toward ontology—specifically, how objects and subjects are transposed and how these transpositions are enacted and enacting. Mol explains:

I privilege practices over principles. . . . This turns doing anthropology into a philosophical move. . . . The ethnographic study of practices does not search for knowledge in subjects who have it in their minds and may talk about it. Instead, it locates knowledge primarily in activities, events, buildings, instruments, procedures, and so on. Objects, in their turn, are not taken here as entities waiting out there to be represented but neither are they the constructions shaped by the subject-knowers. Objects are—well, what are they?13

Below, we describe our methods to investigate the primary objects in our analysis: networks, infrastructures, and algorithms. As will become evident, our approach must grapple with the uneasy tension between methods that continue to center the human, such as interviews, while problematizing the “larger” infrastructures and networks of education governance. While we are aware of this tension, we are also attempting to illustrate how networks, algorithms, and infrastructures are simultaneously objects and subjects, and how these objects/subjects are enacted through various practices and how these objects/subjects enact practices of governing. We are interested in how these practices of governing can be understood through different types of inventive methods in policy studies, including a focus on mobility and networks. It is to these that we move next.

Mobility and Network Ethnography

Education policy studies locate the Anglo-governance model in broader global neoliberal policy imaginaries and reform movements. The presence of new policy networks and relationalities means that educational policy-making and governance are no longer simply occurring within the prefigured boundaries of the nation-state, but now involve a diverse cast of actors and organizations across “global,” “international,” or “cosmopolitan” policy spaces.

Focusing on new spaces and relations of policy provides valuable insights into changing modes of educational policy-making, as well as raising problems for comparison. As Gorur, Sellar, and Steiner-Khamsi suggest, “The challenges to comparison have been posed not only by an increasingly complex, globalizing world with new networks and new technologies, but by the epistemic complexities of comparison.”14 Bartlett and Vavrus argue that studying context and comparison results in difficulties with delimiting field sites in comparative work. Hence, contextual detail needs to be integrated, without attributing a boundedness to the local. One way to deal with a lack of boundedness is to produce a tracing between structures and polices “across individuals, groups, sites or states.”15 Tracing closely relates to what Sheller and Urry call the “new mobilities paradigm” in sociology and geography.16 The mobilities paradigm “enables the ‘social world’ to be theorised as a wide array of economic, social and political practices, infrastructures and ideologies that all involve, entail or curtail various kinds of movement of people, or ideas, or information, or objects.”17 The mobilities paradigm includes empirical investigations and conceptualizing phenomena of movement and interconnectivity.

A focus on mobilities provides inventive incursions in the methodological landscape. From the mobilities paradigm emerged what Büscher and others call “mobile methods,” which aim to “follow” the movement of an object and of people.18 The “following” approach is part of policy mobility studies, evident in McCann and Ward’s three interrelated mobile policy objects—“mobile policies,” “places,” and “people.”19 While the movement of policy has been the traditional domain of the policy transfer and policy borrowing literatures, there has also been recent work in political geography examining how policies move and are created and re-created, both in place and in the processes of moving.20

In a refrain familiar to those who study globalization and education governance, policy mobility draws on the ideas of policy learning, borrowing, and traveling from the policy sciences. Policy mobility highlights how processes of policy transfer—or “translation” as Peck and Theodore put it—are never entirely reproduced but rather emerge through the “inherent tensions between local specificity and global interconnectedness,” or between (contextual) fixity and (relational) movement.21 The mutually recursive processes of policy translation, involving people and places, help to constitute new conditions of possibility for local policy enactment, while these policies are reconstituted by local conditions. There is, therefore, a constitutive and transformative role played by policy travel, in which “policies are not merely being transferred across time and space . . . [but] their form and their effect are transformed by these journeys.”22 Such an understanding of “translation,” at least in respect of policy enactment, emphasizes the contingency of these processes.

Education policy studies use policy mobility approaches to examine new forms of education governance.23 These approaches can provide a counter to “methodological territorialism” or “methodological nationalism.”24 Policy mobility supplements the focus on the state with further units of analysis, including the global (e.g., international organizations), the city, and other more “local” spaces (e.g., schools). In the policy mobilities literature, the central problem is that “policy making has to be understood as both relational and territorial; as both in motion and simultaneously fixed, or embedded in place.”25 This speaks to the ways in which the politics and operations of datafication and governance cannot be understood only within the framework of national borders.26 Neither can they be adequately examined in reference to notions of scale (e.g. local, regional, national, and global) or related processes of the rescaling of statehood.27 This “leaves us grappling to understand what a politics that moves beyond such limitations might look like, how it is constructed, [or] how it is assembled.”28

Additionally, movement and translation occur not only across social spaces but also across sociotechnical spaces made up of human and nonhuman actors. The idea of sociotechnical spaces of policy mobility has been explored within work on new technology and policy.29 Hartong highlights an increasing focus on novel policy mobilities that “include the expansion of standardisation instruments, new monitoring and assessment strategies, or the enhancement of competition and (quasi)market structures.”30 This perspective on sociotechnical mobility draws attention to how people and ideas connected to AI move from laboratories across education policy contexts. We can begin to ask where policy is being made, for example, when AI is used to inform the education policy work of state level bureaucracy, drawing on data analytics provided by business intelligence systems developed in the United States, but also combining expertise from data scientists and input from local policy stakeholders in schools.

Policy mobility, thus, focuses on both multiple relationships inherent in the movement and connections of policies, people, and places and the various discursive and material flows these connections make possible. This includes not only the actors and organizations that constitute these new policy mobilities and networks, but also the types of relations that exist within and across them, and how these connections produce topological spaces in which new modes of policy-making and governance can be effected. It is this relationality that has been the focus of our ethnographic work.

Conceptualizing Infrastructure

Our data infrastructure project initially aimed to focus on policies and the political environments that led to increased use of data in schools and systems. However, as the research progressed, we became aware that the initial problem—concerning whether similar types of policies such as those associated with international large-scale assessment led to similar or different types of data use and data infrastructures—seemed to be important only in combination with complex assemblages of computing hardware and capacities, data literacy, people, dispositions, and technical experts.36 What became evident was the interconnection of infrastructure, algorithms, and policies, and how the multiple sites of our fieldwork were projects “designed around chains, paths, threads, conjunctions or juxtapositions of locations . . . with an explicit, posited logic of association or connection among sites.”37 The shift in focus required new forms of conceptualization and investigation. It became clear that we were studying the conceptual and empirical formation of particular sites of sociotechnical work, and this involved investigating “events and actualizations that take place in a certain field.”38 The projects became studies of new policy spaces as part of the growth and development of infrastructures, and the introduction of algorithmic and automated decision-making.

The field of infrastructure studies gathered momentum with the rise of the internet and associated developments in computing and information systems during the 1990s.39 Infrastructures studies emerged from science and technology studies, including actor-network theory, and from a concern with the role that practices of classification, standardization, and networking have played in the development of modern science. Across a range of fields in the social sciences and humanities, the concept of infrastructure is used to examine the cultural and technical underpinnings of contemporary life.40 Given the diverse sets of networks that constitute infrastructures, and the diverse set of conceptual tools available to analysts of infrastructure, Larkin argues “that discussing an infrastructure is a categorical act. It is a moment of tearing into those heterogeneous networks to define which aspect of which network is to be discussed and which parts will be ignored.”41 Theorizing infrastructure with the aim to understand what it is—the ontology of infrastructure—is a project in pursuit of a general account. As Levinas has shown, this is a pursuit that characterizes philosophy more broadly, which tends to reduce the singularity of exterior things by capturing them under a general concept.42

We set out to follow a different path. We assume that infrastructures are large and complex systems that exceed comprehension from a single vantage point. As such, data infrastructure exceeds its material instantiations in hardware and software; it is also constituted from, and constitutes, social relations, desires, and beliefs. This characterization informs the empirical work in the following chapters. Stories told about infrastructures reflect the multiple and disparate situations from which infrastructures are experienced and the multiple and disparate things and events from which infrastructures are constituted. Rather than tearing into infrastructure to make sense of it, we add our narrative to those told by others involved with particular infrastructures-in-the-making. This perspective allows us to examine claims about technological determinism, hyperbolic AI, and the corporate takeover of education governance. We begin by listening to the stories that technical actors in technology companies, education data analysis centers, and government departments tell about connections between interoperability, standards, and the role of the state in the creation of new infrastructural objects that become part of market exchange. It is through these stories that we can begin to connect infrastructure to governance.

Bevir and Rhodes argue for an interpretivist approach to studying governance that draws attention to the situations and beliefs of individuals from which political and social structures emerge. This is a bottom-up approach that foregrounds the contingency of governance. Stories are situated interpretations that play an important role in this approach, as both objects of study and modes of representation.43 We must attend to both general dispositions and specific situations as part of infrastructure and governance, without subsuming the latter under the former.

Situations refer to contexts in which people act and interpret the meaning of their actions and contexts. As Boltanksi writes, a situation comprises a connection between a context and action, where the meaning of a context is derived from the action.

One and the same context can therefore be the site of different situations, at successive moments, but even, particularly in the case of disputes, at the same time for different actors.44

Our aim is to offer a view of infrastructure and governance as constituted from multiple dynamics across multiple scales. And as we explore in the following chapters, we seek to examine what agency looks like in automated infrastructures, where “infrastructure and software combine as technologies of governance” and create new modes and logics of governing.45

We also draw on the work of Easterling to explore the disposition of infrastructure and new forms of governance, or what Easterling calls “extrastatecraft”—a way to conceive how governance operates through interorganizational networks and a range of actors that include, but also exceed, the state. Easterling shows how infrastructure enables sometimes concealed activities that can be unconnected and/or complementary to forms of recognized statecraft.46 Easterling defines disposition as “the character or propensity of an organization that results from all its activity.” Disposition is a tool for analyzing how infrastructures enable extrastatecraft.47

Extrastatecraft entails a fluid, diffuse, nebulous, and inchoate politics in and of infrastructure, in which disposition acts as a diagnostic of the “accidental, covert, or stubborn forms of power—political temperaments of aggression, submission, or violence—hiding in the folds of infrastructure space.”48 In a critical theory move, Easterling argues that reading disposition involves seeking out the hidden by distinguishing between “what the organization is saying and what it is doing . . . the difference between a declared intent and an underlying disposition.”49

Easterling’s concept of disposition can be traced back to the work of Francois Jullien. As Lloyd explains in the translator’s introduction to Jullien’s book The Propensity of Things: Toward a History of Efficacy in China. Jullien employs the French term dispositif strategique to refer to “how things are disposed strategically so as to be effective.”50 This notion is present in other philosophical developments of the term, of which Foucault’s work is perhaps the most prominent example. Foucault describes a dispositif as a system of relations established between heterogeneous elements.51 A dispositif is primarily strategic and, as Agamben argues, realizes the activity of governance by producing the governed subject. A dispositif is “a set of practices, bodies of knowledge, measures, and institutions that aim to manage, govern, control, and orient—in a way that purports to be useful—the behaviors, gestures, and thoughts of human beings.”52 Infrastructure can be understood as an apparatus in the Foucauldian sense that an “apparatus is a kind of formation” that works not by achieving specific goals, but through the ways in which a political rationality is reinforced.53

A dispositif involves “functional overdetermination,” or logistical power, which produces effects that resonate with one another to rework the system in ways that generate new political rationalities and strategic capacities.54 Bailey applies this concept to the analysis of education policy, showing how “policy refers not only to formal codes and directives from a central authority, but also to a multiplicity of ‘material’ and ‘technical’ forms.”55 With this genealogy of the concept in mind, disposition produces the subjects that find themselves situated within infrastructures, understood as particular arrangements of things and capacities for thought and action. Disposition affords a perspective on how the social traditions and material environments from which infrastructure is constituted act as substrates for, and modes of, governance. 

The subjects produced by dispositions interpret and act upon the situations in which they find themselves, narrating their activities in ways that influence disposition in turn. Easterling suggests, “The stories that a culture tells about infrastructure space can script the use of that space. . . . Some social stories play an additional, powerful role in the ongoing process of shaping disposition.”56 Similarly, Larkin argues for the importance of “being alive to the formal dimensions of infrastructures, understanding what sort of semiotic objects they are, and determining how they address and constitute subjects, as well as their technical operations.”57 These objects include the ways in which markets for data management shape the changing value attributed to data, such as the value attributed to “exhaust data” that “are inherently produced by a device or system, but are a by-product of the main function rather than the primary output.”58

We suggest that analyzing the role of data infrastructure in education governance requires attention to both concepts of dispositions and situations in order to analyze macrolevel dispositions, on the one hand, and what Bevir and Rhodes call “situated agency,” on the other. Therefore, dispositions frame and constrain situations but “no practice or norm can fix the ways in which people will act, let alone how they will innovate when responding to new circumstances.”59 While there has been a tendency in infrastructure studies to focus on logistical power produced through impersonal technical operations, which are often hidden or “black boxed,” the importance of analyzing how infrastructure is represented and interpreted in stories, and as situations, needs to be recognized.

Investigating Infrastructure and Algorithms

Our methods are firmly grounded in the nondigital. By way of contrast, Burrows and Savage argue that the realm of social research has expanded well beyond the academy, to the point that academic sociology and attendant methods (interviews, surveys) are, at best, peripheral. Relatedly, sociology has become a form of nonmethodological synthesis, where methodological innovation—crucial to the development of the field in the mid-twentieth century—is increasingly outsourced to commercial organizations.60 Conversely, Burrows and Savage posit that new approaches to understanding the social world, such as data science, “offer the possibility of describing the social world in a manner hitherto impossible,” while most sociological methods are based on reliance of “accounts of action” in which “data are based on a sample of research participants providing discursive accounts of some prior actions.”61 We recognize this methodological shift but also remain convinced that there is an important role for nondigital, qualitative work in the studies of infrastructure, algorithms, and AI.

Star contends, “We need to be able to theorize across the continuum of information infrastructures, from the old, historical, global to the everyday, simple and quintessentially invisible stuff of ordinariness.”62 A particular methodological approach has emerged from infrastructure studies, which focuses on the empirical significance of banal, everyday processes, reflecting Star’s “call to study boring things.”63 As Star goes on to suggest, “Many aspects of infrastructure are singularly unexciting,” such as the design decisions that go into creating data infrastructures.64 Star also emphasizes the incompleteness of infrastructure, in that infrastructure is constantly being built, maintained, and repaired, while highlighting the ad hoc aspect of infrastructure development and operation, which often only become evident when infrastructures fail.65

One way to examine infrastructure is to map out the technical details, of which there are reams of documentary evidence, requiring at times specialist understanding of fields such as information studies. We focus on aspects such as standards and interoperability in the chapters that follow—technical details that end up being governing tools that span different temporalities and spatialities. We examined publicly available texts of various types, including technical and internal strategy documents and websites. Additionally, we undertook interviews with managers, policy makers, users and technical workers in schools, school boards, ministries, and nongovernmental organizations in Australia, Canada, Japan, and the United States. These interviewees included technical experts involved in building infrastructures, and experts in software development for education governance products such as student information systems.

We also undertook empirical work on the introduction of AI into education governance, in three different ways. First, through identifying the introduction of products that are underpinned by AI, such as student information systems with machine learning–supported computer vision. We focused on the introduction of AI-supported products in education, including the role of technology corporations, and the links to new forms of education governance. However, these new products are also linked to existing modes of education governance and school-based practices to try and redress a gap in understanding “what happens when these systems are used on a daily basis in varied educational contexts.”66

Second, we focused on the users of AI systems to investigate algorithms, premised on the notion that all technology is connected to politics and culture. We follow Seaver, who proposes this:

Algorithms are enacted by practices which do not heed a strong distinction between technical and non-technical concerns, but rather blend them together. In this view, algorithms are . . . unstable objects, culturally enacted by the practices people use to engage with them.67

We undertook interviews with data scientists and information management experts in a data analytics center within a large education department. We asked them about their use of AI and machine learning, including the use of proprietary systems supported by AI, such as business intelligence software. We were interested in what these experts thought AI did, and could do, in education governance.

Third, we undertook interviews with computer scientists who researched and built AI systems in AI laboratories in Canadian, Australian, and European universities. These computer scientists identified with a range of fields, from knowledge-based systems (expert systems) and constraint recognition to neural networks and machine learning. These interviews were used to get “inside” the machine of nascent automated decision-making in education. Getting inside the infrastructures of machine learning has been primarily discussed as the problem of the black box of machine learning and algorithmic decision-making. We are acutely aware of the limitations of interviews as a method to explore automation, and, indeed, as Amoore posits, it is fallacy that we can “break open” black boxes at all, insofar as this assumes that “there is an outside to the algorithm—an accountable human subject who is the locus of responsibility, the source of a code of conduct with which algorithms must comply.”68 Nonetheless, despite the limitations of breaking open black boxes, we were curious to see what is possible to find out about infrastructure and governance, including forms of automation, through interviews. As Edwards notes, to understand the opacity of algorithms and AI means

that comprehending algorithmic systems requires backing out from a narrow focus on algorithms and data per se to a broader frame . . . , or at the very least takes seriously the anthropology of human software developers.69

Overall, we do not claim that this book is based on extensive “in-situ” fieldwork of AI laboratories or applications.70 However, we are interested in Rabinow’s experimentation with fieldwork using an “interview-oriented form of inquiry” that “privileges extensive interviewing with a distinctive group of actors, within a restricted field setting.”71 We have aimed to approximate what Rabinow and Marcus describe as the aim of extended interviewing with technical experts. As Marcus comments to Rabinow, in relation to the attempts of anthropology to be relevant to researching “the contemporary,” “What you have to do these days to pursue successful fieldwork is to locate and construct partnerships—usually unstable—with technicians of general ideas.”72 This has been our approach across multiple projects—to try to create unstable partnerships in relation to ideas—in our attempts to provide accounts of AI, infrastructures, and data. We aimed to create relationships by asking how participants conceptualized their work and their fields. Additionally, our work is designed to not represent our participants as objects of analysis and ourselves as independent from the dynamics of their work, but rather to emphasize different forces shaping them and us. Moreover, we recognized others’ “constitutive enactments” and at times challenged these enactments through our interviews and postinterview work—“we raise questions about it, we doubt it.”73 We are acutely aware of the contrast between what we are investigating—such as new forms of technological decision-making—and how we are doing it. Nonetheless, while we are not creating a new type of method, we are trying a different approach to our investigations through a conceptual register.

Problematization

Studies of governance have analyzed how it emerges, functions, and is narrated in specific cases. As noted above, these studies can be conceived broadly as kinds of “macro” analysis that map how governance moves, the kinds of global networks it uses and forms, and the kinds of infrastructures it depends on and nurtures. Additionally, studies of governance have analyzed how it impacts “local” or “micro” instances of its effects/affects, including all sorts of ad hoc, disparate, uncoordinated, but highly influential impacts on people and systems. In our view, studies of governance should be informed by the view that Deleuze and Guattari celebrate in Gabriel Tarde’s microsociology—that is, the importance of paying attention to “miniscule bureaucratic innovation”74 and the ways in which it is not origins but the types of relations that may, or may come to, matter.75 What follows in the empirical chapters are attempts to do a “fieldwork in philosophy” in order to study nascent forms of automation and data infrastructures in education governance. In each chapter we try out new concepts to understand and create new questions about the hybrid form of synthetic governance. This is based on the premise that our conceptual tools and methods play a part in not only providing parameters for what we will investigate, but “they actively shape and create new realities.”76 Our approach in the following chapters is to construct a conceptual inventory and experiment with these concepts in ways that illustrate how (1) networks, algorithms, and infrastructures contribute to the governance of education; and (2) what sorts of possibilities within these arrangements may be created by automation. Finally, we ask: What sort of politics is possible in contexts shaped by synthetic governance?

Our analyses do not aim to evaluate the efficacy and efficiency of data-driven changes to governance, and the implementation of these changes, in order to address existing problems. Rather, we aim to focus on ontological and epistemological multiplicities, and the possibility of doing governance differently. To orient our analysis, we draw on Foucault’s notion of problematizations, a concept and method that has been used previously in the studies of policy and governance.77 Problematization emerged from Foucault’s project to shift a history of ideas to a history of truth, understood as “a historical space of conditioned contingency that emerges in relation to . . . a more general situation” and that dealt with “vectors” of politics, economics, culture, and science.78

Problematizations are a method for examining the way “practices . . . render something an object of thought (as moral reflection, scientific knowledge, political concern), thereby enabling it to enter the play of truth and falsehood.”79 Problematizations focus on specific situations, in which there are “institutionally legitimated claims to truth,”80 and contestation and provisional settlement around the acceptance and authority of different forms of governance. Foucault suggests that simultaneity is the condition of problems and solutions; for any problem there are multiple responses that can be and are proposed. Problematization “develops the conditions in which possible responses can be given; it defines the elements that will constitute what the different solutions attempt to respond to.”81 Problematization, as a method for studying the contingency, rather than inevitability, of datafied governance can help us to force new kinds of empirical and conceptual engagement—premised on multiplicity—with data, infrastructure, and algorithms. In this engagement, we aim to unsettle existing responses to infrastructure and algorithms in education.

While we recognize that Foucault was hesitant about ascribing notions of change or transformation to problematizations, we are interested in Stengers’s engagement and extension of Foucault’s ideas. This is about, as Stengers notes, considering “problematization as a transformative engagement, an engagement which forces the thinker to test the limits of thinking.”82 The limits of thinking mean that we can force disruptions of existing problems to create other ways of asking questions about contemporary governance. Problematization, therefore, is not just a mapping method, but rather an experimentation in order to understand changes in thought and to unsettle common sense:

Showing that things are not as obvious as they seem is trying to render people unable to continue thinking them the way they did, to make it harder for them to stick to positions they had considered uncontroversial or common sense. When this happens, a transformation becomes possible.83

We take problematization as a license for theoretical experimentation in conjunction with the different empirical examples. Our discussions are grounded in empirical ethnographic work but, importantly, are also motivated by “trying to appropriate conceptual tools that had been forged for certain problems, and to refashion them in the hope that they will provide analytic purpose for different problems.”84 Our aim then is to examine what new problematizations are formed within the following examples of datafication and automation in education, and the possibilities for new conceptualizations of nascent phenomena in education governance.