Given the theoretical nature of this contribution, the majority of this section uses prior theoretical literature to articulate these qualities. In many cases, such prior work provides valuable conceptual vocabulary for describing various parts of these qualities and how they operate. At the same time, few if any of these concepts alone help us account for the whole. Put differently, this section shows how the combination of qualities characteristic of subjectivities enlivened within and around algorithmic systems requires the multiplicity of theoretical perspectives marshalled here.
4.1 Inferring
Inference plays a key role in algorithmic systems. A tuned content moderation system can infer which content is likely to be flagged by users as objectionable. A trained topic model can infer the topics present in a novel document. Numerous potentially sensitive attributes about a social media user can be inferred from seemingly benign information about them [
97,
170]. This ability to make inferences has significant consequences, as described in the case studies above. Such inferential predictions, we suggest, are a defining characteristic of how algorithmic subjectivities are done.
Prior work [e.g.,
30,
77] has highlighted how the informational activities of labeling and classifying are both epistemic acts of knowing—what type of thing is this data point?—and simultaneously exercises of political power. Put differently, classification places the data point into an existing knowledge structure with its own history, value commitments, political underpinnings, and so on. When the thing being classified is a human person, especially when the person is being classified by someone else or something else, the act of classification becomes an exercise of power, insofar as it defines that person in ways that may or may not align with how they define themselves.
However, algorithmic inferences go beyond purely classifying or labeling data points, human or otherwise. Indeed, the subjectivities enlivened through algorithmic systems may not have previously existed as a class or label. Returning to our case studies, a blogger may, by writing in a particular way, indicate implicitly or indirectly that they hold a specific type of critical stance on the autism criteria in the DSM. While it would certainly be possible to label a blogger as being critical of the DSM criteria without topic modeling—based on, say, explicit statements they make—topic modeling works to enliven that subjectivity of criticality in a particular way. This algorithmic enlivening simultaneously casts the subjectivity as manifest through statistical patterns in word choice, suggests that observations of an individual’s language use may allow for inferring the degree to which the individual performs that subjectivity, and may overshadow or even exclude other possible interpretations of that same language. In such ways, the interplay of algorithmic systems goes beyond assigning existing categories—anti-vaxxer, troll [
47], victim [
96], and so on. Instead, they work to enliven subjectivities that are seemingly familiar yet simultaneously predicated upon mechanistic inferring [cf.
58].
Those mechanistic inferences are often based not only on directly observable data but also on a so-called latent feature space [e.g.,
123]. In the above ASD case study, each topic (described by a probability distributions over words) is treated as one dimension of such a latent feature space. The words in these probability distributions, due to their semantic meaning, are readily human interpretable. For instance, in the example from the case study above, a topic’s high probability words (
spectrum,
disorder,
autism,
diagnosis,
disorders, etc.) can be interpreted as providing a brief description of what the topic is about. Similarly, representing a document in terms of these latent topics also provides a human interpretable semantic description about the content of those documents. The dimensions of this space, i.e., the topics themselves, are referred to as “latent” because they are
inferred, rather than being directly observed in the data.
Although many algorithmic systems employ such latent feature spaces, they are not all as readily interpretable. For instance,
Large Language Models (LLMs) [e.g.,
63,
123] use latent dimensions that often lack any obvious or semantically meaningful description. Instead, representations of words and documents are transformed into an embedding space, sometimes referred to as a “semantic space,” often comprised of a few hundred dimensions. This size contrasts with traditional approaches to document representation, which often have one dimension for every word in the vocabulary (i.e., tens of thousands of dimensions). Thus, although they may use more latent dimensions than is common with topic modeling, these embedding spaces still offer a significant reduction in the number of dimensions used to represent a document. The mappings, from words or documents into these latent representations, are inferred by iteratively optimizing their use in predicting masked tokens (i.e., a single word that has been omitted from a sentence) [
63] in massive textual data sets (e.g., web crawls from millions or billions of web pages). The resulting representations often significantly reduce overall sparsity, which in part accounts for these representations enabling improved performance on downstream NLP tasks (sentiment analysis, named entity recognition, text classification, etc.). At the same time, though, the use of such representations significantly reduces the ability for a human to determine what any given latent, i.e., inferred, dimension means.
This difficulty in interpretability of inferences has a number of consequences. For instance, it partially explains why so many different techniques have been developed simply to detect the presence of biases in LLMs [e.g.,
28,
84,
140,
143], i.e., because the limited interpretability of the semantic representation space makes biases difficult to notice. It also helps account for ways that online advertisements can be targeted to, for example, cannabis users, even when an advertising platform does not offer cannabis use as an explicit targeting option [
29; see also
125,
170]. Put succinctly, algorithmic systems make inferences based not only upon statistical patterns within observed data but also using transformed representations of those observed data that are themselves inferred and thus often not readily human interpretable.
At the same time, algorithmic subjectivities are not comprised solely of data points to be analyzed or of models to be manipulated. Consider, for instance, the processes of enlivening autistic subjectivities through assemblages of the DSM, clinical practices, educational institutions, government bureaucracies, and so on, one cannot “run” this subjectivity like a model to make predictions about the likelihoods of different outcomes, such as how various treatment options might influence an individual’s eventual educational attainment, annual earnings, or life satisfaction. Similarly, one would not be able to predict automatically the goodness-of-fit between this subjectivity and any given person. Thus, algorithmic subjectivities do not, per se, make predictions or inferences about the world. However, the capacity for making quantitative predictions—for inferring—becomes a distinguishing characteristic of algorithmic subjectivities when that capacity becomes entangled within broader assemblages of persons, institutions, and structures of meaning, as described further below.
4.1.1 Designing for Inference.
These inferential capabilities connect with challenges in designing algorithmic systems [
69,
114,
197,
198]. Designers must somehow anticipate not only the results that are algorithmically surfaced during any interaction but also the broader sociotechnical ecosystems in which that content may emerge. Furthermore, the algorithmic models themselves change in response to ever changing data streams.
Two strands of thought are helpful in understanding and designing around such inference. The first comes from work on modernism [
159,
189], which is often described as having four key tenets: calculability, efficiency, predictability, and (hierarchical) control (for a concise description, see [
35], p. 950). Inference is intimately connected with each of these: to make inferences, bodies must be made calculable; inferential models enable making predictions about a person’s actions; and so on. Thus, understanding algorithmic subjectivities requires understand how bodies come to be calculated, to be made efficient, to be made predictable, and to be (hierarchically) controlled.
Put differently, seeing algorithmic systems as a modernist enterprise helps guide our analytic and design attentions. Consider two examples, both drawn from the above case study about topic modeling and ASD. On the one hand, topic modeling translates different perspectives (e.g., criticality toward the DSM-5 revisions) into something that can be calculably identified, in this case, by examining statistical patterns of word co-occurrence. These inferences simultaneously enable making predictions about how individual bloggers might behave or react to certain events (e.g., future DSM revisions) [similar to.
86,
153]. On the other hand, the use of such inferential predictions enables a host of design possibilities, ranging from tools that individual bloggers could use to reflect upon their own family’s journey via analysis of their writing (similar to [
32]) to systems that posit connections among multiple blogs for various purposes (similar to [
20]). Designers can attend to these core tenets of modernism by considering what things are (and what things are not) made calculable, made efficient, predicted, and controlled by different design possibilities. Doing so provides a conceptual language to account for the various ways that algorithmic system design might figure in enlivening different subjectivities.
Second, we can also understand the distinct role that inference plays by drawing on the notion of the scalable subject [
172]. Described as a refinement of the data double [
87], Stark [
172] highlights how digital traces about an individual are used to create mathematical and computational models. These models and their attendant uses represent a unique confluence of work in the psychological sciences and in computer science, one that has significant ramifications for the understanding of, and for the control of, individual persons.
To illustrate these points, Stark draws on a variety of examples, from A/B testing to Facebook’s emotion contagion study [
108], to mental health tools intended to assist patients with mood and behavior disorders (e.g., Ginger.io). These cases and others, he argues, all involve assuming that relationships between different variables that occur in the aggregate will also apply to those same variables for an individual (citing the “ecological fallacy,” [
144]). Put differently, scalable subjects are created in part by when observations of past data points (often humans and their activities) are used to draw conclusions about new data points, i.e., to draw inferences about them. It is such inferences that allow for the scalable subject’s scalability.
Thus, we suggest, this conceptual apparatus [
172] is useful for reasoning about, and perhaps for designing around, the means and consequences of inference. For instance, it is perhaps obvious that the content moderation of posts and individuals—as bully, target, toxic, bystander, and so on—occurs via algorithms making inferences. The conceptual lens of scalability suggests that designers attend to the mechanisms by which inferences are made. For content moderation, a given post would first be projected into a latent feature space, as described above. Then, the post’s distance
5 within that feature space could be used to infer how likely the post is to be an instance of, say, bullying or toxicity. The use of this inference mechanism is based on an assumption of scalability: that the manifestation of toxicity results in posts that, when projected into this feature space, become geometrically proximate. Focusing on the mechanisms by which these inferences happen allows designers to consider whether such scalability should hold in this context, or if there might be other means for identifying toxic or bullying content.
While analytically useful for examining, and perhaps for designing around, inference, the notion of the scalable subject [
172] provides less guidance or insight about individual subjective experience. Indeed, “lost in descriptions of the aggregate are the ways in which individual subjects understand their own scalability” [
172, p. 213]. As we have argued here, though, algorithmic subjectivities involve not only individual humans’ subjective experiences but also their interplays among multiscalar actors. Thus, understanding algorithmic subjectivities requires complementing the scalable subject with other theoretical devices that account for such entanglements.
4.2 Entangling
Algorithmic subjectivities are enlivened via interactions among and within heterogeneous assemblages of actors and processes. Thus, we do not claim that algorithms enliven subjectivities on their own. A moderation system may flag a charged piece of content; a topic model may assign a poignant topic to a document. Yet such computational procedures do not function as entirely independent, distinct entities, neither analytically nor practically. Rather, the subjectivities in which we are interested arise through algorithms that are enmeshed or entangled [
13,
79] in heterogeneous aggregates of entities, actions, and interpretations. Such subjectivities come to be enlivened through the entangling of, for instance, content moderation algorithms within processes that interweave human users, automated systems, policy-forming bodies, low-wage human labor, norms of acceptable interpersonal interaction, and so on. Thus, it is not the algorithm
per se that enlivens algorithmic subjectivities, but how an algorithm operates as part of and comes to be entangled in what are always much broader structures. Indeed, this entanglement is concomitant with the distinctly massive scale and scope at which such subjectivities operate.
Such entangling raises at least three interconnected concerns for which we need to account. All three, in various ways, involve questions of how individual actors interact with one another within these entangled assemblages.
First,
whither agency? Put differently, if individual actors can no longer be seen as entirely distinct entities, how can we conceive of the agency with which individual actors act? Consider a treatment from Latour [
112] of competing claims related to gun regulation. One side claims that “guns kill people,” while another side claims that “people kill people; not guns.” These two statements offer competing claims about agency (and, thus, responsibility). Latour resolves the apparent tension by arguing that neither statement is entirely accurate. Instead, by entering into relation with one another, the gun and the person holding it combine to become a different kind of actor, “a citizen-gun, a gun-citizen” [
112, p. 32]. In this way, “it is neither people nor guns that kill;” instead, “responsibility for action must be shared among the various actants
6” [
112, p. 34].
Similar logic can be applied to understand agency within algorithmic systems. As an example, Stochastic Gradient Descent (SGD), a common algorithm for training ML models, becomes a different kind of actor when applied to data in which harassing or toxic content has been labeled. To be sure, content moderation systems do not kill anyone in the same way that a gun does. However, content moderation systems do play a role in flagging content and banning users. To say that the system (or even the trained model on which the system is based) does the banning or flagging elides the significant complexities involved. The trained model, the SGD algorithm, the training data, the human content moderators on whose labor those training data are based, the programmers who build the system, the (often corporate) organizations who collect these data and oversee these systems—none of these is solely responsible, because agency does not belong to any single one of these entities. Rather, as Latour suggests, responsibility must be distributed among them.
The notion of distributing responsibility raises a second question: given this entangled soup of actors,
how might we go about defining individual entities? If responsibility is distributed among the “actants,” as Latour [
112] calls them, how do we even determine who or what these actants are? Is it reasonable even to consider doing so? One approach to these questions draws on the notion of entanglement from Barad [
13]. She argues that these entities do not necessarily exist as such prior to their interactions. Rather it is through what Barad calls intra-actions that entities become co-constitutive of one another. As an illustration, quantum physics dictates that it is possible to know either a particle’s position
or its momentum, but not both simultaneously. Barad points out how this account reinforces a duality between world (i.e., the particle) and representation (i.e., measurements of its position, momentum, etc.). Put differently, this problem posits that there exists a particle that has both position and momentum. Instead, Barad suggests that, prior to observation, “the particle simply did not exist in any fixed state” [
99, p. 929], but rather existed in a state of indeterminate potentiality. This is not to say that the particle does not exist at all prior to observation, but rather that aspects of its existence were as yet indeterminate. Through the phenomenon of observation, the particle, its position or momentum (but not both!), the measurement apparatus, and the knowing human observer all come to be in a particular stabilized relation. Barad refers to this as an
agential cut, the moment/process by which indeterminate entities come to be stabilized (even if only temporarily) into mutually constitutive relationships.
Recent work, especially from Frauenberger [
79], has suggested that entanglement theories offer a novel generative metaphor for HCI. Entanglement, it is argued, provides a fundamental reconceptualization of the “interaction” in HCI. “Things and people, as phenomena, mutually constitute each other through their intra-action, i.e., the boundaries between human and machines are not pre-determined, but enacted” [
79, p. 9]. Just as the particle is brought into a certain fixed state (and excluded from other states) through its intra-action with a measurement apparatus, a human observer, and so on, any given sociotechnical system exists in a state of indeterminacy until entangled in some particular way. Frauenberger [
79] illustrates this point using the example of
Flow, a “hypothetical device [that] displays the ease or anxiety of members of a conversation based on data from a range of sensors” [
79, p. 12]. Different means of evaluating this system enact different agential cuts: “an interview study will make Flow a cultural artefact, a controlled user-testing study in the lab will make it a functional tool, and a long-term diary study might make it an artificial sense of people” [
79, p. 15].
This kind of thinking helps us account for the complex relationalities within algorithmic systems. The bully, the victim, the toxic content, the onlookers or bystanders—all of these come to be entities with certain properties and relations among them because of the agential cuts made by content moderation systems. Again, this is not to say that, for example, the bystanders did not exist prior to content being flagged. The bystanders were there, but they came to take on the role and enter into the relations of being bystanders with certain properties (and not other properties they could have had) in part because of the content moderation system’s intra-actions with them and with the other actors involved.
If we apply these notions of indeterminacy to algorithmic systems, a third question arises:
how might we conceive of power dynamics and the exercise of authority? Couldry and Mejias [
53] argue that the collection and analysis of data operate in power dynamics that resembles colonialism. Much as colonial governments appropriated land, bodies, and natural resources to maximize profits, technologies companies analogously quantify social interaction into data to extract value from it. Much as capitalism has historically transformed human activity into the commodity of labor, data colonialism transforms the experience of human sociality into the commodity of data. Thus, in this formulation, technology companies who collect and analyze these data become the dominant enactors of data-driven authority.
Similarly, Burrell and Fourcade [
38] suggest that the power to make such determinations resides with those whom they call the “coding elite.”
“The coding elite is a nebula of software developers, tech CEOs, investors, and computer science and engineering professors, among others, often circulating effortlessly between these influential roles [….] Most valued in this world are those people who touch and understand computer code. Most powerful are those who own the code and can employ others to deploy it as they see fit.” [
38, p. 217]
If “code is law” [
115], the argument goes, then those who write the code make the laws and, thus, have the power to govern.
However, when considered through the lens of entanglement, the coding elite do not occur as a pre-figured entity. Rather, this group comes to be constituted in this way because of its interconnections within much broader systems, of “start-ups, […] large firms, government-sponsored research labs, classrooms,” and so on [
38, p. 217]. Put differently, an entanglement approach suggests that the coding elite do have significant power, but that that power does not occur solely because of their ability to write (and deploy) code, nor because of their ability to collect data and extract value from it [
53]. The coding elite come to be the coding elite because of their intra-actions with algorithmic systems, where design and implementation are a form of intra-action. The coding elite, the bystander(s), the moderation systems themselves—each is mutually co-constitutive of the others and their attendant properties, including power differentials.
Thus, questions of power are not simply a matter of who gets to make the agential cuts. We should not say, for instance, that the coding elite are responsible for creating bystanders (or other subject positions) of online harassment. Instead, we should ask how conditions of possibility are shaped. Recall that, according to Barad [
13], before they enter into mutually co-constitutive entanglements, entities reside in a state of indeterminacy. That indeterminacy, though, should not be conceived of as a uniform prior,
7 so to speak. Put differently, a given entity is not equally likely to be constituted as, for example, either a bully or a victim in a given situation. Rather, the particulars of the operant sociotechnical assemblages (weighting of feature vectors, platform moderation rules and policies, training data sets, content flagging by individual users, etc.) shape whether certain entities are more or less likely to be co-constituted in certain ways.
Accounting for power dynamics, then, requires examining how current configurations (of bodies, technologies, organizations, etc.) work to make various future configurations more or less likely. For instance, given a particular content moderation system (and its attendant antecedents, as described above), who is more likely to be constituted as the bully, the bystander, the victim, and so on? And what processes operate to increase or decrease these likelihoods? Achieving the accountabilities necessary to address such questions requires, among other things, strategies by which we might use notions of entanglement to inform algorithmic system design.
4.2.1 Designing for Entanglement.
To return to the thinking we introduced earlier, subjectivities here are not references to internal or mental concepts of the self, the individual; they are not born solely of one’s emotional encounters with, say, a topic model, a content moderation system [
181], or a Facebook post [
36]. Rather, we use the term “algorithmic subjectivities” to highlight the ways transient or mutable bodies come into being through the affective push and pull, the affective attunements [
100], algorithms make possible. For instance, the controversies surrounding the classification of autism in the DSM-5 undoubtedly trigger highly personal subjective responses. As suggested in the topic modeling case study described above, the families of those living with autism are deeply affected by the subtle changes in text across different versions of the DSM.
Our point, however, is that as algorithms assume increasingly prominent roles within these assemblages, a commensurately algorithmic nature arises in the attendant subjectivities that are probable or even possible. The calculative operations of, for instance, the topic model, translating and enumerating the content that people write, provide an affectual register for understanding the DSM controversies. Through their linguistic interactions, the actors across a discussion platform both work to define and come to be defined by the calculative potential of particular topics as well as by how affectual or subjective responses can be computed across these topics. The attention we give to algorithms in enlivening subjectivities then is due both to the sheer scale of the assemblages they are coming to constitute and to the power they exert as part of such entanglements [
5].
The conceptual and theoretical considerations described above also suggest paths to adapt our design methods. Consider, for instance, personas and scenarios [
42,
50,
134,
146]. Typically, a persona is defined in terms of some combination of individual attributes: demographics, professional goals, personal interests, educational background, family and personal connections, and so on. As an example, Nielsen [
134] describes a set of personas related with electronic health care, one of which is “Gitte”:
“Age 40. General Practitioner. Mother to two kids, age five and seven. Married to an academic. Lives in a larger provincial town. Works in a shared [medical] practice. […] Attends a choir once a week and jogs in a sports club regularly. Is member [sic] of a book club. Has a conservative/minimalist attitude towards technology and professional tools in general. […] She is smart, lean and takes care to be dressed in well-designed clothes, exquisite jewellery, and newly cut hair.” [
134, pp. 178–179]
Such descriptions implicitly focus the designer’s attention on attributes of the individual. Alternatively, we could articulate a persona less in terms of its individual attributes and more in terms of its entanglements. What are the technological, political, cultural, economic, social, and other entities with which the individual in this persona is being entangled? What are the heterogeneous collections of intra-action by which these interweavings occur? What other possibilities existed for this actor prior to their these intra-actions? How is this persona both subject to and constitutive of these entanglements? In this example, how is “Gitte” as an actor mutually constitutive of the practice where she works or the choir in which she sings? Such an approach would likely require us to articulate not only a series of individual personas but also the relationships among them. Similarly, the scope of a scenario shifts from a particular interaction to a particular intra-action, that is, the agential cut(s) by which the entities involved come to take on their state as those particular entities (and not other types of entities) with those particular properties and relationships (and not other kinds of properties and relationships that might have been possible).
Going further, the notion of entangling could be used to disrupt the very idea of centering in design. HCI is often described in “user-centered” or “human-centered” terms [
17,
75,
152,
177]. The conception of algorithmic subjectivities contributed in this article offers at least two distinct opportunities for future directions of work. First, designers could resist such centerings from the beginning and instead explore entangling as the conceptual locus of design, or what might be called “entanglement-centered” HCI [cf.
79]. Second, designers could resist the concept of centering altogether. Selecting any given concept or entity—humans, users, entanglement, and so on—as the center of design implicitly de-emphasizes other potential centerings. Instead, the notion of entanglement could provide a path for pursuing what might be referred to as “uncentered” design, one that gives primacy to no single entity or concept. Either of these possibilities resonates with other calls for applying more-than-human-centered approaches to design [
184,
185].
In saying this, we do not suggest that humans are unimportant. Rather, we suggest that designers explore shifting their focus away from specific attributes of individual entities and toward systemic entanglings. In doing so, the designer’s questions change from “What tasks are made easier or more difficult by this design?” to “What unique entanglings are made more or less probable by this design?”; from “How useful, pleasurable, confusing, rewarding, etc. are users’ interactions with this design?” to “How could this design play various roles in entangling users within broader sociotechnical structures and systems?”; from “How will this design decision cause users to react?” to “How will this design decision enable or preclude different ways of being?” Again, we are ambivalent as to whether entanglement should lie at the center of design processes or should be one consideration among many in a potentially uncentered design.
In either case, such shifts may help resist the impulse to see humans (or users, or communities, or whatever else is being centered) as prefigured entities. That is, rather than taking the human for granted as an entity around which our design centers, an emphasis on design as entangling draws attention to how the design—both as a rendered technical artifact and as a discursive process—plays a role in co-constituting the humanity of those humans. Such points hint toward another potentially fundamental shift, one that pertains to the “human” in HCI.
4.3 Humanizing
As described throughout this article, HCI has developed progressively more nuanced ways of talking about humans and their interactions with computers [e.g.,
17,
75,
163,
177]. At the same time, as noted above, human-centered approaches implicitly posit “human” as a prefigured category. In contrast, the perspective advanced in this article suggests attending to the processes by which the categories of human, algorithm, and so on come to be.
A variety of prior work has suggested that the ways a machine “thinks” fundamentally differ from the ways that a human “thinks,” or at least that their capacities for and processes of “thinking” differ in scale and affect [
2,
176]. For instance, Burrell [
37, p. 6] illustrates how a “neural network [trained to recognize digits in human handwriting] doesn’t, for example, break down handwritten digit recognition into subtasks that are readily intelligible to humans, such as identifying a horizontal bar, a closed oval shape, a diagonal line, etc.” Rather, the visual features to which the network attends appear entirely alien and nearly unrecognizable for a human viewer (
Figure 1). Burrell acknowledges that “there is certainly a kind of opacity in the [largely subconscious] human process of character recognition as well” [
37, p. 7]. That said, very few of us, when attempting to decipher a number written by someone with poor handwriting, are likely to say that we look for patterns resembling the amorphous blobs in
Figure 1.
The case studies above suggest similar examples. For instance, topic modeling represents relationships among topics as mixtures of probability distributions, but this representation bears little resemblance to the ways a human reader might encounter themes within a corpus of documents [see also
156,
179]. Likewise, content moderation on discussion forums and social media platforms may rely on the coupling of humans and algorithmic processes, but such a coupling can sometimes produce frictions. For machines, the application of standards and policies turns on thresholds. In this way, sand dunes can be flagged as nudity [
128], and photojournalism can be flagged as child pornography [
178]—the kind of error that is more likely in purely algorithmic moderation than with human moderators. In contrast, humans are compelled to read into and respond to the accounts of people’s lives, such as personal interactions with a user guiding exactly what punishment moderators choose to dole out [
169].
At first glance, it may seem like such examples demonstrate two clear categories, human and machine, each with their own styles of thinking. Instead, we suggest that the functioning of algorithmic systems works, almost paradoxically, to define what constitutes the category of human. This defining happens in at least two ways. First, these algorithmic systems both encode and enact definitions of “human.” Prior work focuses primarily on how this enactment happens discursively. For instance, Keyes [
106] provides a valuable investigation into AI research on diagnosing autism, which often treats social behavior and communication skills, or perhaps an inferred lack thereof, as indicative of autism. Put differently, “autism is treated as oppositional to the traits that ‘make’ a person a person” [
106, p. 14]. Thus, they argue, these systems encode a formulation of autism (and of humanity) in which “autists are portrayed as asocial, fundamentally lacking in the ability to know and understand, and consequently, lacking in agency and personhood” [
106, p. 3].
We suggest that similar logics operate both in the technical implementation details and in the practical functioning of algorithmic systems. The Facebook user is defined as much via their interactions with entities on the platform (including friends, organizations, advertisements, etc.) as via the aggregation of their “Likes.” Similarly, techniques such as recommender systems encode the user as a mostly economic actor seeking to maximize selection of objects in a way that aligns with their tastes and preferences [
168]. These and other algorithmic systems both construe (and continually reconstrue) what it is to be human.
Second, the very notion of AI posits particular types of intelligence as artificial [
23,
110,
124,
158]. Put differently, this kind of terminology suggests a distinctly artificial, i.e., algorithmic way of thinking. By labeling it as such, we implicitly define human thinking (and being) as different from, or perhaps even as opposed to, algorithmic thinking (and being). Thus, the manner in which these systems operate works to define what is human via counterexample.
This is not to say that the distinction between algorithms and humans is unnecessary. Put differently, we are not all homogeneous objects [
94] or uniform actants within a network [
113]. Instead, rather than accepting these as prefigured categories, the position on algorithmic subjectivities advocated here suggests that we—HCI researchers, designers, practitioners, and so on—should attend to the ways that these categories are done, to the processes by which distinctions come to be made between algorithmic and human.
4.3.1 Designing for Human Meaning.
In these ways, the functioning of algorithmic systems is deeply, inextricably intermeshed with the human values designed into and interpreted from them [
22,
23]. Akin to the cyborg from Haraway [
92], the resulting assemblages are a hybrid creation, at once both familiar and strange. They incorporate elements that, on their surface, appear familiar to us—language use, demographics, social connections, and so on—but within algorithmic contexts, those elements take on subtly different meanings and significances. For Haraway, and for other accompanying and subsequent work in post-human or more-than-human scholarship [e.g.,
31,
100,
175], this hybridity invites a situated reading of meanings, values, (subjective) bodies, and their entanglements. For example, patterns of language use—certain colloquialisms, specific racial slurs, or even particular combinations of emoji—could increase the likelihood that a given piece of content is flagged by a content moderation system [
95,
166]. Despite their surface similarity, such features have different meanings for a content moderation classifier and for a human reading (or writing) the content in question. What a human sees as connotative of harassment, an algorithm encodes as weights within a feature vector.
Topic modeling, from the first case study above, offers a prime exemplar. At its core, a topic model is based on counting. The model’s probability distributions are fitted to a data set based on patterns in the numbers of times that groups of words co-occur in documents together. To be sure, numbers matter—numbers count, so to speak. At the same time, it can seem almost ridiculous to assert that the most frequently occurring word in a document is also the most important word [
141]. Indeed, the single most common word in almost any English document is “the” [
56]. This disconnect between frequency and importance contributes to the use of
stop word lists [
149]. These lists include words that occur so frequently as to convey almost no meaning in and of themselves, so they are simply omitted or “stopped out” [
119, p. 27] when processing a document. Although such lists are more common and/or more copious with some techniques than with others [
119, p. 27], the use of stop words reveals a fundamental divergence: between the frequency counts and probability distributions of topic modeling, and the meanings and significances ascribed by human readers [see also
155].
Furthermore, computational implementations of topic modeling treat words not as meaningful
per se but simply as unique tokens. For a topic model, the words “autism” and “autistic” are just as different as the words “apple” and “orange.”
8 An occurrence of the word “autism” is only meaningful in so far as it affects the inference of the model’s underlying probability distributions. That is, a fundamental disconnect occurs between the computational processing of this language and the human interpretation of it.
To facilitate design processes, it may be productive to foreground such disconnects. For instance, when designing an interactive system built upon topic modeling [e.g.,
20,
66], incorporating topic modeling results into early prototypes can help designers, as well as users or co-design participants, interpret the model’s results. At the same time, this strategy can give the impression of a system that has some level of human-like understanding about the relationships among words within a topic, such as the “autism” and “autistic” example above. As an alternative, such words could be augmented with the random token ID number that a model assigns to each word, for example, “token2475-autism” and “token1382-autistic.” Doing so offers a simple means to highlight the differential between computational representation and human interpretation, while still enabling productive feedback during design iterations.
On one hand, the specifics of such differentials occur in particular ways for topic modeling. On the other, this same point applies, albeit with different technical details, to other computational approaches for processing natural language.
For instance, during the time that this article was being finalized for submission, language models informed by distributional semantics [
26,
27] and attention mechanisms [
183] were developed to address some of the very issues described above. In such LLMs and their applications—perhaps most notably ChatGPT [
139], but also including compositional word embeddings [
123] (e.g., word2vec), bidirectional encoder representations from transformers [
63], other generative pre-trained transformers [
148], and so on—the representation of an individual token depends on the context it which it appears, i.e., the other proximate tokens. Furthermore, the tokens in these representations are usually not comprised of natural language words but of subword segments. Returning to the above example, the word “autism” might be represented as two tokens (e.g., “aut” and “ism”), while the word “autistic” might be represented as three tokens (e.g., “aut,” “ist,” and “ic”). Thus, and in contrast to traditional topic modeling approaches [
24,
25], such language models often provide very similar representations for such word pairs. This feat is accomplished by leveraging, in addition to subword tokenization, moderately high dimensional so-called semantic spaces, similar to the latent feature spaces described above [
63,
123]. In such approaches, an individual word token (or a subword token, or a complete sentence, or an entire document, etc.) can be represented as a vector in that space. The similarity between any two word tokens, subword tokens, sentences, documents, and so on is then based on the cosine of the angle between their vector representations. These representations are derived from training the model on large corpora (usually billions of words from combinations of web crawls, books, and other texts), such that words occurring in similar contexts will have representations in similar regions of the semantic space.
At first glance, this general approach takes clever advantage of the assertion from early Wittgenstein [
193] that a word only has meaning within the context of some statement, some logical proposition. Later Wittgenstein [
194], however, developed the notion of language-games, asserting that a word (or a complete sentence, or an entire document) derives its meaning from use in the context of some human activity. As a simple example, the single-word sentence “Fly!” has drastically different meanings when uttered by an irritated customer lifting their glass of Chablis, by the pitcher on a baseball diamond, or by a grey wizard dangling from a precipice. This point goes beyond simple polysemy or word sense disambiguation (for which some computational approaches have been developed [e.g.,
85,
130,
154]), and beyond the fact that the other surrounding words describing each of these situations will differ. The point is that the processes by which humans actively work to construct and contrast the meanings of these words [
151] likely bear little resemblance to computing the cosines of angles between vector representations based on subword tokens. Yet it is those distances between feature vectors that can make the difference between whether or not a given piece of content is flagged for moderation.
Thus, future work that seeks to advance our understandings about experiences around algorithmic systems must explicitly consider the relationships among the pluralities of meaning in these systems, the computational mechanisms that help give rise to those meanings, and the sources of agency involved with enacting those meanings [
15,
65,
165,
195]. As Barad [
13, p. 353] puts it, “phenomena—whether lizards, electrons, or humans—exist only as a result of, and as part of, the world’s ongoing intra-activity, its dynamic and contingent differentiation into specific relationalities.” None of these phenomena, entities, and so on can be reductively treated as standing alone or viewed from outside their contexts. Rather, they must be understood as always becoming through their entangled relations. Our analysis here focuses on the specific context of algorithmic systems, and how these complex relations and interplays—among probability distributions, feature vector weights, data point labels, exercises of power, attributions of meaning, and so on—afford a distinct role in co-producing the category of “human.”
Attention to such dynamics is equally important for researchers investigating algorithmic systems and for designers implementing algorithmic systems. As researchers, we should pause before interpreting these systems’ constituent elements in a manner similar to the varying manners in which we humans might interpret those same elements in other, non-algorithmic contexts. For instance, word-based features that are highly informative for a toxicity classifier do not necessarily have the same meanings and connotations in the context of that classifier as when a human reads those same words in a social media post. As designers, we can and arguably should attend to the ways that the internal functionings of the algorithmic systems that we implement work implicitly to define what constitutes humanity. For instance, the features used to curate a social media news feed not only can influence perceptions of how close we are to specific individuals [
72] but also can also work to reshape how we perceive the constitution, enactment, and performance of human closeness.