24 Technosocial Scotomas in the Algorithmic Age

Sociotechnical Entanglement

Jasanoff’s (2006) coproduction framework emphasizes that science, technology, and society are mutually constitutive and deeply intertwined, with social systems and technological designs influencing and reinforcing each other. This framework reveals that technologies do not develop in isolation; rather, they are embedded within social, cultural, and political contexts that shape their design, implementation, and outcomes. Failing to recognize this dynamic can obscure how technologies and societal structures coevolve and perpetuate existing inequalities, leading to technosocial scotomas. When examining the impact of technology, we must also consider the specific context in which it is used (Suchman 2007) and the active role of the observer or user in constructing meaning from their interactions with technology (Crary 1990). This process of adaptation and coevolution between users and technologies can lead to unforeseen outcomes that extend beyond the original intentions of the technology’s designers, illustrating the complex interplay of agency and structure in knowledge production and technological deployment.

The shift from objectivity to operativity in knowledge production, as Halpern (2015) discusses, plays a key role in the emergence of technosocial scotomas. Our current data-driven models and algorithms prioritize action, control, and prediction, which can obscure the limitations, biases, and value judgments embedded within these systems. This operativity, Halpern argues, reflects a broader transformation in how knowledge is produced and used—not just for understanding the world but for managing and controlling it. The aestheticization of data, where complex systems are made visually appealing and appear objective, further masks underlying power dynamics, ethical concerns, and issues of validity. By focusing on clean, efficient representations, these systems can obscure their role in reinforcing social inequalities or perpetuating existing power structures, thus creating scotomas in our understanding of their broader implications (Halpern 2015). Following Ernst’s (2013) media archaeology, technologies are not just representations but operations, temporal machines that structure and manipulate space-time. In this view, technosocial scotomas are not only sociopolitical absences but temporal compressions and occlusions created by algorithmic systems. These systems reframe perception by privileging immediacy and prediction, masking the contingencies and historical depth of sociotechnical contexts.

Actor-Network Theory (ANT) highlights how technosocial scotomas arise when we fail to recognize technologies as active participants within complex networks, producing unintended consequences (Latour 2007). Technologies are not merely passive tools but actants with their own agency, shaping social relations and outcomes in often unpredictable ways. This distinction between actants (i.e., nonhuman entities with effects) and agents (i.e., with intentionality) is crucial. Although algorithms lack human intentionality, they nonetheless exercise significant impacts within sociotechnical networks, often exceeding their designers’ intentions and control.

As Latour argues, agency is distributed across both human and nonhuman actors, meaning that these scotomas are not simply human oversights but are coproduced through the dynamic interactions between social forces and technological systems. In contrast, the concept of unintended consequences typically focuses on unforeseen outcomes of human actions, often treating technologies as neutral artifacts. The scotoma framing, grounded in ANT, emphasizes how technologies have agency within these networks, and can unintentionally reinforce or create new social dynamics through their design, deployment, and use, contributing to emergent forms of inequality or exclusion. While ANT proposes a form of analytical symmetry between human and nonhuman actors, this doesn’t imply an ethical or political equivalence. Humans retain distinctive responsibility as the architects and governors of technological systems, even as these systems exercise increasing autonomy and influence within networks.

The Case of inBloom

The rise and fall of inBloom serves as a cautionary tale illustrating the technosocial scotomas that can arise in the development and deployment of educational technologies into the marketplace (Dieterle and McWilliams 2022). Despite its initial success, inBloom quickly faced backlash from parents, educators, and privacy advocates concerned about the collection and potential misuse of sensitive student data (Singer 2013). These concerns were exacerbated by poor communication of inBloom’s value proposition, lack of transparency, and insufficient engagement with stakeholders, particularly parents. As news of inBloom spread, growing public concerns about privacy, security, and the commercialization of student data led to a loss of trust in the initiative (Simon 2013). The backlash was intensified by the perception of inBloom as a top-down initiative imposed on states and districts, as well as the initiative’s struggles with its ambitious scale, complex state-level partnerships, and the mismatch between its agile startup approach and the slower pace of educational bureaucracies (Bulger et al. 2017).

As political pressure on governors increased, participating states began to withdraw from the initiative within months of its public launch. By April 2014, inBloom had lost support from all participating states and was forced to announce its closure, just thirteen months after its high-profile debut. While inBloom failed as an organization, its brief existence catalyzed important national discussions on student data privacy. This led to over four hundred new pieces of state-level legislation, industry commitments like the Student Data Privacy Pledge, and increased awareness of the need for transparent data practices in education (Bulger et al. 2017).

The imposition of inBloom’s after-effects on education policy was not merely a result of poor execution. It was emblematic of deeper, systemic technosocial scotomas that continue to challenge the integration of technology into all levels of education globally. Table 24.1 provides a concise summary of the six key manifestations of technosocial scotomas associated with inBloom: (a) overlooking social impacts; (b) assuming more data leads to better outcomes; (c) ethical blindness and privacy concerns; (d) interacting cultural and economic scotomas; (e) political ramifications and democratic governance; and (f) the hidden human labor, energy, and environmental costs. By systematically confronting these technosocial scotomas, we can better work toward more equitable, ethical, and socially responsible development, deployment, and management of algorithm-driven tools and platforms in education.

Critical Constructionism

One key factor contributing to the development of technosocial scotomas is the unequal distribution of social, economic, and political power and influence in the development, deployment, and governance of technological systems (Bijker et al. 2012). The Social Construction of Technology (SCOT) framework highlights how various social groups shape technologies, with dominant groups often exerting disproportionate influence on the design and outcomes of technological systems. The scotoma framing extends beyond missing information to encompass power and exclusion in knowledge production and technology design and deployment, a concept Feenberg (1999) describes as the technical code, wherein the design and structure of technological systems embody and reproduce dominant social values and power relations. While blind spots often refer to gaps in an individual or organizational perspective, technosocial scotomas emphasize systemic and structural oversights arising from hegemonic control and privilege. Such control, often exercised by elite groups, can lead to the prioritization of their interests and values at the expense of marginalized communities, reinforcing existing social inequalities, and obscuring the needs and perspectives of those most affected by the consequences of these technologies (Crawford 2021; Muldoon et al. 2024; Reich et al. 2021).

Under algorithmic conditions, transmission models of education are technologies designed for doing things better. These models, characterized by Freire (2000) as banking models of education, and by Papert (1993) as instructionism, emphasize the direct transfer of knowledge from knowledgeable sources to less knowledgeable recipients to improve instructional efficiency, accelerate content delivery, optimize systems, forecast outcomes, and automate decisions. Without critical reflection, however, they perpetuate legitimate knowledge (Apple 2004) and the hidden curriculum (Giroux and Penna 1979), reinforcing and potentially amplifying biases, power structures, and inequalities based on class, race, gender, and ability (Dieterle et al. 2024).

In contrast, turning to a kind of critical constructionism, when engaging with algorithms, holds promise for addressing technosocial scotomas. Rather than doing things better, critical constructionism is about doing better things with technology. Critical constructionism emphasizes the constructionist philosophy of knowledge building through the creation of digital and physical artifacts with and for others to solve local issues and global challenges. This approach engages learners in questioning and understanding the societal, ethical, and cultural implications of the technologies and solutions they develop. Critical constructionism presents a pedagogy for understanding and addressing technosocial scotomas by (a) learning and making critically and (b) working with objects-to-think-with-critically. Critical constructionism acknowledges that technologies inherently contain built-in limitations and unforeseen consequences, which Simondon (2017) identifies as constitutive of technological evolution itself. Rather than assuming these can be eliminated through better design, this approach encourages learners to anticipate, identify, and develop governance approaches for these inevitable uncertainties and failures.

As an example of what I have in mind, consider a critical constructionist STEM curriculum where AI ethics is integrated into course materials, and students are invited to investigate the limits of machine learning models. Building on previous work in critical mathematics education (CME), such as that of Rico Gutstein (2013), this approach would have students use these models as objects-to-think-with-critically, probing outputs to uncover technosocial scotomas. Noticing that their models disproportionately flag low-income and minority defendants as high-risk, they could dig into the training data, revealing historical biases in policing and sentencing. Teachers of such curriculum would have read Simone Browne’s Dark Matters: On the Surveillance of Blackness (2015) and Ruha Benjamin’s Race After Technology: Abolitionist Tools for the New Jim Code (2019). Their students would come to realize exactly how their prediction models could amplify inequities if used in the criminal justice system. Student efforts at designing alternative recidivism risk assessment models would reveal the ever-persistent emergence of technosocial scotomas or what Ole Skovsmose (2005) called “the ethical erasure” of mathematical modeling. Throughout the project, students would engage in cycles of ethical reflection, technical iteration, and critical dialogue with each other and guests of the course (e.g., members of affected communities), coming to see that the models they investigated and are designing are not just neutral tools but sociotechnical systems embedding value judgments with real-world consequences. Through such inquiry, students develop the epistemic humility necessary to recognize technosocial scotomas as recurring, rather than eliminable, features of sociotechnical systems.

While critical constructionism cannot eliminate all technosocial scotomas, as these are partly constitutive of technology itself, it provides a framework for making them visible, contestable, and governable through democratic processes rather than allowing them to operate as new hidden curricula within educational systems. It empowers learners to actively shape the development of emerging technologies in more ethical, equitable, and socially responsible ways, addressing the technosocial scotomas that transmission models of education risk perpetuating. As we navigate the challenges and opportunities of the Algorithmic Age, critical constructionism offers a pedagogy for cultivating the critical consciousness and agency necessary to steer technological development, deployment, and use toward beneficial ends.

References