by Luke Munn
What happened at the home of James Bates in Bentonville, Arkansas on November 22nd, 2015? On that Saturday night, Bates invited a few of his friends around to watch football and drink. But the next morning Bates called 911 and police arrived at the house to find one of these friends, Victor Collins, dead in the backyard. Bates claimed to have let his friends crash the night and fallen asleep around 1am, before waking to find the body himself and make the emergency call. But the police suspected a homicide as Zusanna Sitek and Dillon Thomas reported at the time:
“After getting consent to search Bates’ home, detectives found Collins floating face up in the hot tub and noticed the water was tinted red and appeared to contain bodily fluids and blood… They also noticed Collins had a black eye, a cut on his eyelid, his lips were swollen and bruised, and he was bleeding from his mouth and nose.”1Sitek, Z., & Thomas, D. (2016, February 23). Bentonville PD Says Man Strangled, Drowned Former Georgia Officer. Retrieved from http://5newsonline.com/2016/02/23/bentonville-pd-says-man-strangled-drowned-former-georgia-officer/
Bates was known to have several smart home devices, including a “Nest thermostat, a Honeywell alarm system, [and] a wireless weather-monitoring system”2Wang, A. (2017, March 9). Police land Amazon Echo data in quest to solve murder. Chicago Tribune. Retrieved from: http://www.chicagotribune.com/bluesky/technology/ct-amazon-echo-murder-wp-bsi-20170309-story.html. But the device that most interested the Bentonville Police Department was his Amazon Echo. Physically, the Echo is understated: a black cylinder that internally houses a set of seven microphones and an audio speaker. Yet this unassuming form belies its real significance as a gateway for Alexa, Amazon’s digital assistant that is “always listening”—capable of isolating speech within a room, deciphering it into a textual phrase, transmitting it to the “cloud,” and responding appropriately by answering questions, relaying news, playing music, activating lighting, and so on.
What kinds of conversations and activities did this device log on the evening in question? To follow this lead, the police seized the Echo and issued a subpoena to Amazon for all the data associated with that particular account, suggesting that they had “reason to believe that Amazon.com is in possession of records related to a homicide investigation being conducted by the Bentonville Police Department”3 Wang, A. (2017, March 9). Police land Amazon Echo data in quest to solve murder. Chicago Tribune. Retrieved from: http://www.chicagotribune.com/bluesky/technology/ct-amazon-echo-murder-wp-bsi-20170309-story.html. At the time this request was rejected by Amazon, which argued that both these voice requests and the responses given were protected as free speech under the First Amendment4State of Arkansas vs. James A. Bates, No. CR-2016-370-2 (Benton County Circuit Court February 17, 2017). Retrieved from: https://assets.documentcloud.org/documents/3473799/Alexa.pdf.
Yet art is not limited by fact, but can speculate about fiction. The artwork constructed here, Monitor, thus acts as if this informational request was accepted, using “smart home” notifications to extrapolate from the evening’s events: a few friends, a few drinks, a floating body. The artwork takes the form of a webpage that can be run in any modern browser. Upon landing at the page, visitors encounter a simulated desktop with wallpaper, icons and a clock widget set at 6pm on November 22nd. As the evening progresses, notifications appear in the top right, logging a series of activities: a garage door opened, a phone call made, a television switched on, a cocktail mixed. The details are carefully sourced, from the phone numbers of local businesses to the basketball games airing that day on television. The time period in question can be played through at an accelerated rate, or “scrubbed” back and forth, allowing events to be examined and replayed. Though simple in conception and delivery, these mechanisms allow every visitor to have a unique experience that highlights particular notifications. Rather than forcing a narrative, Monitor, like many artworks, maintains a certain ambivalence or ambiguity, allowing each viewer to derive their own understanding of the evening’s events.
Of course, in terms of content, there is an unavoidable darkness to such a narrative. Blood spatters and broken glass, intoxication and strangulation—these kind of images run counter to what Florian Cramer, calls the “glossy innovation narratives”5Cramer, F. (2014, April). Post-digital research. Presented at the Transmediale 2014, Berlin, Germany. Retrieved from: https://www.youtube.com/watch?v=3ciJtJUwcug usually associated with such products, undermining the relentlessly positive, life-enhancing imaginaries of Silicon Valley tech titans. There is a certain attraction to such morbid tales, and some of the press received, thus far, by the artwork, reflect this fascination. One article from Fast Company was titled “The First Murder Mystery For The Smart Home Age6Wilson, M. (2017, May 3). The First Murder Mystery For The Smart Home Age. Retrieved April 30, 2018, from: https://www.fastcodesign.com/90116588/the-first-murder-mystery-of-the-smarthome-age”; another piece in the German edition of tech blog Engadget was similarly named “Webprojekt Monitor: Ein Smart Home beobachtet einen Mord7 Knoke, F. (2017, April 3). Webprojekt Monitor: Ein Smart Home beobachtet einen Mord. Retrieved from: http://de.engadget.com/2017/05/03/webprojekt-monitor-ein-smart-home-beobachtet-einen-mord/” (Web project “Monitor: Smart Home witnesses a Murder”). But, while the content of this artwork may carry some frictional capacities, the project is equally interested in the form that such content takes. How do such devices index, filter, and frame the world, producing particular formations of knowledge?
The “smart home” is typically championed as the rationalization of household activity, the epitome of Le Corbusier’s maxim that the “house is a machine for living in” that will usher in a new “order of work and leisure”8Le Corbusier. (1986). Towards a new architecture. New York: Dover Publications. (p. 95, 102). In this imaginary scenario, the smart home is a self-adjusting, self-regulating system, incessantly logging previous activities, daily cycles and yearly trends. This data underpins the calibration of living conditions for maximum comfort and convenience—controlling climate, dimming lights, adjusting music, locking doors. Domestic life is algorithmically optimized. But these “common-sense” operations are also epistemological, forming a knowledge-structure that apprehends the environment and its inhabitants in a certain way. It is through “mechanisms of inscription, recording, and calculation,” Lucas Introna asserts, that “algorithmic actors emerge as producers of particular domains of knowledge.”9Introna, L. (2016). The algorithmic choreography of the impressionable subject. In Algorithmic Cultures (pp. 38–63). Routledge. (p. 27) What does this knowledge say? Data often promises to discard awkward ambiguities and messy subjectivities and replace them with hard empiricism. As Chris Anderson once quipped: “Who knows why people do what they do? The point is they do it, and we can track and measure it with unprecedented fidelity. With enough data, the numbers speak for themselves.”10Anderson, C. (2008, June 23). The End of Theory: The Data Deluge Makes the Scientific Method Obsolete. Wired. Retrieved from: https://www.wired.com/2008/06/pb-theory/ The notifications of smart home devices provide an access point into this particular logic, one which “reads” the world – and is, in turn, “read back” by users, institutions, and state-actors – in a particular way. Above all, Monitor attempts to explore this particularity.
Yet we might also think about the “dumb home” (in a non-pejorative sense)—about the ways in which certain aspects or activities escape registration. What qualities leak out of the epistemological categories established by the smart home and the wider Internet of Things? Firstly, the smart home itself has a certain porousness, and some activities are captured far beyond the boundaries of a house or property. In the case of Bates, for example, a significant piece of evidence came not from his personal devices, but from the Bentonville electric and utilities department, which recorded a massive amount of water used in the middle of the night. According to their data the “residence used 50 gallons of water between 1 to 2 a.m. on Nov. 22 and an additional 90 gallons of water between 2 and 3 a.m.”11Sitek, Z., & Thomas, D. Against the imaginaries of the smart home and the focus of many privacy advocates, such activity was registered on a regional, not a personal level through the decidedly less “innovative” infrastructure of utilities. But if some activities are “re-captured” beyond the home, some disappear altogether. The lawyer for Bates, Ms. Weber, disputed the prosecutor’s allegation that the deck was hosed down in order to remove evidence of a homicide. As reported in The Information “in a decidedly non-digital strategy, she says the water outside the tub couldn’t have come from a garden hose. Mr. Bates had a salt water tub, and she says all the water on the outside of the tub had salt residue.”12Dotan, T., & Albergotti, R. (2016, December 27). Amazon Echo and the Hot Tub Murder. Retrieved from: https://www.theinformation.com/articles/amazon-echo-and-the-hot-tub-murder Of course, this hard dichotomy between “digitality” and “materiality” is a false one. As work by scholars like Friedrich Kittler, Matthew Kirschenbaum and Jussi Parikka have shown, our digital infrastructures, from the cables and data centers of the “cloud” to the coltan within cell phones, are all too material. Nevertheless, the salt water observation reminds us that there are certain capacities of matter, certain aspects of subjectivity, and certain everyday activities that remain unregistered by the smart home and, more broadly, by the supposedly ubiquitous technical regimes that surround us.
So what does all this mean for contemporary arts practice? On the one hand, the regimes underpinning the smart home are highly scaled, dispersed throughout infrastructures which reach across the boundaries of nation-states and the borders of territories. As Erich Hörl suggests, this is a “culture of control that is radically distributed and distributive, manifest in computers migrating into the environment, in algorithmic and sensorial environments.”13Hörl, E. (2017). Introduction to General Ecology: The Ecologization of Thinking. In E. Hörl & J. Burton (Eds.), General Ecology: The New Ecological Paradigm (pp. 1–74). London: Bloomsbury Academic. (p.4) Even the simple act of locating one’s self with a smartphone, for example, encompasses the device in a hand, the rare earth minerals that comprise it, the undersea cabling of the internet, the array of satellites needed for geospatial positioning, and a data packet pinging through to a server. Indeed Alexa is only made alive by the transmission of data throughout Amazon Web Services, a network of 54 zones sprawling across 18 regions: Beijing, Mumbai, São Paulo, Seoul, Sydney, Frankfurt, London, amongst others14Amazon. (n.d.). Amazon Web Services (AWS) – Cloud Computing Services. Retrieved from: https://aws.amazon.com/. Undoubtedly, then, there is value in artists becoming more technically literate, in understanding some of the sprawling complexity of informational infrastructures and thereby “making the invisible visible”15Bridle, J. (2015). Exposing the Invisible. Retrieved from: http://exposingtheinvisible.org/films/group/james-bridle/. Here one might think of the work of James Bridle, Julian Oliver, Trevor Paglen, Danja Vasiliev, and many others whose practices might fall under the loose umbrella of “tactical technology” or “critical engineering.” For these practitioners, “you can’t critically engage with technoculture and its infrastructure if you’re unable to unravel its threads, run your fingers through the seams, [and] visualize its jurisdiction”16Politics, A. (2013, November). The Weekest Links : #stacktivism. Stressfm. Lisbon. Retrieved from: http://blog.stress.fm/2013/11/the-weekest-links-stacktivism.html.
But, on the other hand and, arguably, more overlooked, are the ways in which these regimes move not just outwards but inwards: permeating into the inner life of the individual, the inner space of the kitchen or living room, the inner qualities of affect or emotion. Such power is effective precisely because it is not overt and exterior, but rather penetrates into the psychic fabric below. This capillary power, as Foucault put it, “reaches into the very grain of individuals, touches their bodies and inserts itself into their actions and attitudes, their discourses, learning processes and everyday lives”17Foucault, M. (1980). Prison Talk. In C. Gordon (Ed.), Power/knowledge: Selected interviews and other writings, 1972-1977 (pp. 37–54). New York: Pantheon. (p. 39). Rather than spectacular, this power is subliminal. This interior space presents a new and highly profitable terrain for colonization by capital. Amazon, for example, recently filed a patent application for a “voice sniffer” algorithm that could be configured to detect so-called trigger words, “a verb indicating some level of desire or interest in a noun,” such as “I like skiing,” or “I love product X”18 Edara, K. (2017, November 9). 0170323645. Reno, Nevada. Retrieved from: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=1&f=G&l=50&co1=AND&d=PG01&s1=amazon.AANM.&s2=conversational&OS=AANM/amazon+AND+conversational&RS=AANM/amazon+AND+conversational. Language is parsed here not just semantically but emotionally, slowly constructing an intimate profile of aspirations and motivations. Similarly, a Google patent detailed how a user’s mood might be determined by the “volume of the user’s voice, detected breathing rate, crying,” and her medical condition approximated based on “detected coughing, sneezing, and so forth”19Zomet, A. (2016, September 8). 0160260135. Mountain View, California. Retrieved from: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=1&f=G&l=50&co1=AND&d=PG01&s1=privacy-aware&s2=google.AANM.&OS=privacy-aware+AND+AANM/google&RS=privacy-aware+AND+AANM/google. Another Google patent application demonstrates how particular audio signatures can be used to detect activities: the chatter of a family meal, the water flow of toothbrushing, and even childhood “mischief” by correlating the presence of children’s voices with “low-level audio signatures (whispering or silence)”20 Fadell, A., Matsuoka, Y., Sloo, D., & Veron, M. (2016, September 8). 0160261932. Retrieved from: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=3&f=G&l=50&co1=AND&d=PG01&s1=Google.AANM.&s2=mischief&OS=AANM/Google+AND+mischief&RS=AANM/Google+AND+mischief. It is this inner, intimate realm that represents the new frontier for technical regimes, regimes instrumentalized towards the imperatives of capital. If these are the kind of forces that are reconfiguring human experience, then art needs to follow, not merely to observe and critique, but also to move ahead, actively speculating about the possibilities that such conditions enable.