Rhetorical Spaces

A blog about writing, rhetoric, and emerging mobile technologies

Who Owns Augmented Space?


Earlier this month, Snapchat launched an art-based component to its popular augmented reality platform. The project, known as "Snapchat ART," places geo-located art all over the world, allowing users to find and view digital artworks through the Snapchat camera. As of now, Snapchat is in control of the artworks that are generated at each location and individual artists can submit proposed locations and artworks through the platform. For one of its first ART installations, Snapchat partnered with artist Jeff Koons to install a large AR balloon dog in Central Park. Within a few weeks, however, Koons' project had been "vandalized" by local NY artists, who claimed that Snapchat's project raises a number of troubling questions about space, corporate art, and digital publics that need to be addressed as AR technologies become more widespread.

As graffiti artist Sebastian Errazuriz put it in describing his counter-augmentation: "Should corporations be allowed to place what ever content they choose over our digital public spaces? Central Belongs to the City of N.Y. Why should corporations get to geo-tag its GPS coordinates for free?" The Snapchat incident highlights a central question that we will have to wrestle with in the years to come as AR technologies become more ubiquitous: who owns augmented space?

According to Lev Manovich, "every point in physical space can be said to contain some information that can be retrieved" ("The Poetics of Augmented Space, 228). When we walk through a space like Central Park, it is already littered with potentially augmentable objects and spaces rife with personal and historical information, from a bridge where a couple got engaged to a sidewalk where a person lost their life. As many spatial theorists have pointed out, the meaning of a space is determined by the actions, ideas, feelings, and desires of those who inter-act within and through it. Thus, when we ask who "owns" an augmented space, we are really asking who (e.g. a tech company, the general public, a group of people, etc.) has imposed the dominant lens, either officially or de facto, through which this augmented space can be accessed.

Mark Zuckerberg received an appropriate amount of public backlash when he chose to bring his AR "toon" to the flooded streets of Puerto Rico. In what was clearly a marketing move to promote Facebook's new mixed reality platform, Zuckerberg's gaff highlighted how merely "being" (or, in Zuckerberg's case, "not-being") in a space is itself a rhetorical act. By transposing a virtual avatar onto a space whose inhabitants are in immediate physical danger, Zuckerberg (whether intentionally or not) exposes the anesthetized rhetoric through which we experience environmental disaster as a mediated phenomenon. In other words, the AR technology Zuckerberg demonstrated in his promotional video merely magnifies a rhetorical process that occurs when we scroll through our news feed: Much like Zuckerberg's avatar, many of Facebook's 2 billion users were able to be virtually present for Puerto Rico's disaster and yet remain shielded from its material effects.

However, Facebook's PR flop might have just been ahead of its time. Fast forward several decades into the future and digital avatars reporting from disaster areas might be a much more normal sight on social media. Thus, in a sense, the question "who owns augmented space?" has a much simpler, and much more banal, answer than we would like to admit: the usual suspects. Nearly every major tech company is currently vying to become the "lens" through which we view our augmented future, and although AR content will certainly be participatory and user-generated, similar to the way it is on existing social media platforms, we also know that it will be highly surveilled, infested with misogyny and racism, and, ultimately, "owned" and controlled by those who stand to reap the most benefits from it.

When we say that a space is "owned" we often conceptualize it through a discourse of commodity exchange. In the same way that a physical property can be purchased, an augmented property (i.e. a set of GPS coordinates) might be purchased so that a company can claim exclusive rights to augmenting it. Indeed, as Errazuriz laments: "We know they [tech companies] will make money renting gps spots to brands and bombard us with advertisements." Imagine walking through central park with your friends, only to be stopped every five seconds by digital Coke cans and dancing cereal mascots. In fact, this is often the kind of nightmare capitalistic scenario imagined by AR futurists. In his novel Rainbow's End, for instance, Vernor Vinge imagines a techno-dystopian future in which physical space is covered with so much augmented content that the protagonist is forced to drive miles into the countryside before he can escape it.

When it comes to AR then, maybe the concept of spatial "ownership" is the wrong way of thinking about it; rather, maybe augmented space is less about ownership and more about negotiation. As Errazuriz's AR "vandalism" demonstrates, creative (mis)uses of AR platforms is exactly the kind of thing that will shape AR's future as a platform for digital and location-based writing. As Susan Delgrange points out in her book Technologies of Wonder, "digital media are shaped by rhetorical exigency and cultural imperatives" (11). Certainly, artists and digital activists should continue critiquing corporate takeovers of digital space and exposing the inequitable power structures that foster hegemonic mediations of physical space. However, as Madison Jones and I point out in our recent webtext for Kairos, we should also pay attention to how these platforms can be leveraged as technologies for engaging with alternative or undisclosed rheorics of a space, rhetorics that may be operative in our cultural subconscious but need to be brought into more explicit view.

Augmenting History


If you've ever played Tony Hawk Pro Skater 4, then visiting Alcatraz Island in San Francisco is a very surreal experience. The entire time I was touring the island, I had to fight the urge to jump up on the handrails and pretend I was doing a kick flip onto the lower deck. (I can't skateboard at all; I definitely would have broken my neck.) As someone interested in augmented reality, this strange convergence between digital and physical spaces was truly fascinating.

Of course, I didn't spend the entire time at Alcatraz trying to find "S-K-A-T-E." Mostly because my wife and I were utterly engrossed in the free audio tour that came with our admission. As we listened, the nondescript cells came to life as the narrator pointed out the beds occupied by Frank Morris and the Anglin brothers, the prisoners involved in the infamous Alcatraz escape of 1962. The audio tour even included actual recordings from when the prison was still operational. When I walked into the dining hall, I almost jumped when a prison guard from the 1950s shouted at me to "shut up and eat!"

Being at a location with an audio tour is not simply a matter of looking; it's a matter of inhabiting, and the audio supplement works so well at a place like Alcatraz because it allows the prison to remain visually unadorned with distracting signs and images. Unlike stationary placards or historical markers, it allows visitors to move through the space, thus creating less blockage in more popular areas. Moreover, it creates a more immersive historical experience by coating the visual artifacts with layers of aural history.

According to Emma Rodero, audio-based narratives are so compelling because the listener is "constantly building [their] own images of the story in [their] mind." Similar to reading a book, audio narratives are a fundamentally participatory media in the sense that it leaves space for the listener's imagination to participate in the creation process. For location-based tours like Alcatraz, then, the location itself provides a kind of visual and material catalyst to the user' s reception of the audio.

But what about locations (historic or otherwise) that don't have prominent visual elements? The site of the Battle of Gettysburg, for instance, looks like a normal field unless viewed through the augmented reality tour offered by InSite. Creating location-based tours, then, must take into consideration how the space itself has already "written" the visual elements of the scene. If rhetoric is about discerning our "available means" as writers, then we must consider what is (and is not) already available as a rhetorical factor within the location itself.

As John Tinnell points out, physical spaces operate as "techno-geographic interfaces" within AR applications: "AR enables the creation of media that incorporate geographic flows as a vital element in the composition or design process" (78). For Tinnell, writers of location-based projects must maintain a sense of openness or contingency to the "accidents of the sensible" present within a location that will always serve as constitutive rhetorical forces within the user's reception and experience of the project. As the Alcatraz example demonstrates, writers of location-based tours must allow the development of AR content to remain "permeable and transparent" to the location itself (80).

Writing a location is always a form of co-authorship. As augmented reality becomes more advanced and ubiquitous, we must remain careful as writers not to simply incorporate AR media unreflectively into location-based writing projects. Rather, we must carefully consider how the augmentations respond to the rhetorics already embedded within the user's environment.

Pokemon Go to Sacred Spaces


I was in Europe for two weeks this summer. As is probably a common experience for many first-time visitors, I went to more museums, memorials, and churches than I can remember. After a while, I started to notice some interesting variations in how these spaces encode and enforce their historical, political, and/or religious significance by regulating the actions of visitors: at a 10th century church in north Germany, I was told not to film the paintings; at a beautiful, towering cathedral in Poland, I was told to remove my hat; at the former home of Frederic the Great in Potsdam, Germany, I was told to not take any pictures of the palace interior (or else pay ten euros for a "photo pass"); at the Memorial to the Murdered Jews of Europe in Berlin, I wasn't "told" to do anything. With each interaction, I gained a little more knowledge about not just why but how spaces are configured as sacred through the actions of those who pass through them.

Public memorials are often designed to be physically obtrusive for visitors. They are material markers whose physical presence is not only intended to represent but to materially enact collective remembrance. In many cases, memorials are designed to (literally) get in the way of the quotidian activities of public life. The German artist Gunter Demnig, for instance, has been installing "stolpersteins" (or "stumbling stones") throughout Europe since the early 1990's. In what is described on Wikipedia as "the world's largest decentralized memorial," a stolperstein is placed at the last freely chosen place of residency for those sent away to Nazi concentration camps. As memorials, stolpersteins are designed to be intrusive, to carve time out of people's habituated routes as they walk by and "stumble" upon them.

I was on a family trip to New York City recently, and one morning we took a train downtown to visit the 9/11 memorial. Walking down Greenwich Street, it's impossible not to see (and hear) the Twin Tower reflection pools. Erected at the original site of the North and South Towers, the pools operate as cultural pedagogy, or as a material instruction for how we are to respond to 9/11 as an historical event: the dialectic between national suffering and national healing is materially "reflected" by the pools' disappearing center (unspeakable trauma) and flowing water (outpouring of hope and support). Similar to Deming's "stumbling stones," the public is meant to encounter the reflection pools as a physically obstructive reminder of American tragedy, incorporating its message as an embodied, affective event.

Surprisingly, I also felt all of these things when my Pokemon Go avatar came across the North Tower reflection pool.

I think one of the reasons I was so shocked by the digital reflection pools is because of the disruption that the memorial evoked within the procedurality of the game's digital environment. As a Pokemon Go player, I typically proceed un-reflectively through the digital landscape of Pokemon Go, only stopping to ponder its non-descript trees, intersections, parks, and buildings insofar as they participate within the processes embedded within the game (i.e. as locations for catching/fighting pokemon). In the case of the North and South reflection pools, however, their presence within the digital world of Pokemon Go was (for me at least) significant beyond their role within the game’s processes. Although they were just a collection of pixelated blue squares, their presence struck me as unique and significant, sticking out like public scars in an otherwise bustling, energetic landscape. The digital reflection pools were so noticeable because they reversed the procedural logic that many find troublesome in the game's ability to (super)impose itself upon sacred spaces.

For someone like Leonard Pitts jr., I should be “throttled” for even having the application open in such a place. Pitts’ article "Capture This! It's wrong to play Pokemon at Auschwitz!" comes in response to reports that Pokemon Go players have been spotted capturing digital monsters at sacred spaces such as Holocaust museums and national cemeteries. For Pitts and many others critical of the game's careless dispersal of pokemon spawning locations, augmented reality games like Pokemon Go are not only distracting for the other visitors, but more importantly dishonor those for whom the space is intended as a site of remembrance.

Pitts is not alone in his disdain for Pokemon Go. From hospitals to private homes, Niantic has received a plethora of requests from various locations asking to removed from the game. Although it seems simple enough to implement in the next app update, Niantic currently has no system in place (at least to my knowledge) for barring pokemon spawning in locations like memorial sites, museums, and cemeteries. (Although, if you happen to be friends with the Niantic Board of Directors, you can personally request that pokemon be prevented from spawning on private property) But even if Niantic were to implement a system for restricting pokemon spawning locations, it would be nearly impossible for Niantic to remove all unwanted pokemon considering that the designation of a space as "sacred" is determined by a multitude of cultural, ideological, and political factors that are not likely to be taken into account for every square inch of the game's digital landscape. Pikachu will always be trespassing somewhere, it seems.

For the most part, it seems that people are not as upset with Niantic as they are the people who choose to engage with the game in a place like Auschwitz. After all, other mobile games and social media apps still function in such places, and we don’t hold those developer responsible for their user's inappropriate actions. The real culprit, it seems to me, is the game's augmented reality functionality, or the overlaying of digital pokemon within physical space. This feature, even though it is a relatively minor aspect of the game, is often featured prominently in articles discussing this issue. Indeed, it’s hard to deny that a smiling, bouncy Koffing (a pokemon who excretes poisonous gas) at the Holocaust Museum creates a jarring and inappropriate visual juxtaposition. There is a kind of digital colonization taking place that undeniably configures the Pokemon Go player as flippant (at best) and cruel (at worst).

In "Of Other Spaces: Utopias and Heterotopias," Michel Foucault describes how spaces become configured as "heterotopic" when they juxtapose "in a single real place several spaces, several sites that are in themselves incompatible” (6). For Foucault, such spaces are full of "superimposed meanings," which then generate further meanings through their concatenation. As Robert Topinka explains, "by cutting and clashing with order, heterotopias force new forms of knowledge to emerge" (64).

What "new forms of knowledge" emerge in the clash between sacred-physical and playful-digital spaces? When I first saw the integration of the north and south tower reflection pools within the game's digital rendering of my surroundings at Ground Zero, I (literally) confronted the key difference between virtual and augmented reality environments: whereas virtual reality traditionally places the user in an entirely digital world with no necessary correspondence to physical reality, AR must necessarily take into account the physical spaces of the user's surroundings in its design, including those elements that merely serve to be "disrupt" the AR experience. I know it seems simple, but the lesson to be learned from all of this, at least in terms of AR as a technology for writing within public spaces, is that "augmentation" is not simply the (super)imposition of playful digital elements upon physical spaces (e.g. pokemon at Auschwitz), but simultaneously the (super)imposition of sacred physical elements upon playful digital spaces (the Twin Tower reflection pools). Indeed, as AR technology begins to move forward and more GPS-enabled mobile AR games are developed, we are likely to encounter even more "incompatible" juxtapositions between physical and digital space.

Celestial Realities


Augmented reality is on track to become a $120 billion industry by the year 2020, AR startup Magic Leap just received an unprecedented $800 million in C round venture capital investment, and Vuforia, a leading vision-based augmented reality platform, was recently purchased for $65 million by PTC. Without a doubt, the last few years have seen a staggering amount of interest in AR technologies. As a result, some of the smartest people in technology, business, marketing, and design are starting to flock to this emerging industry. But for some reason, they can't seem to convey the potential of this technology beyond kids looking at models of 3D planets.

This might seem like a strange trope, but it's actually more fitting than you think. After all, isn't outer-space the perfect metaphor for augmented reality? Kind of like the "final frontier" of computational advancement? Popular media often uses outer space as a symbol for curiosity, exploration, and territorial conquest, and the child-users in the videos are the future explorers, or inheritors, of this impending technological unknown.

Or, to phrase it less optimistically, this trope exposes the underlying colonialist impulse of "augmented reality" as a mass medium. When I say that AR "colonizes space," I mean it in two different ways: First, AR colonizes space in the more obvious sense that digital advertisements, social media feeds, email reminders, and solar systems will visually populate the physical spaces of everyday life. However, there is another, less obvious form of colonization taking place within AR development.

When Google released it's beta-version of Glass, one of the biggest complaints was that the device was essentially a smartphone that you wear on your face and thus failed to live up to its potential as an AR optimized optical display. Well, Google has responded to its critics by taking on an even more ambitious AR program: Project Tango. At first glance, Tango seems indistinguishable from other AR devices and platforms. I mean, the promo video even features a kid staring in wonder at an augmented solar system! However, if you read up on the technology behind Tango, Google is actually taking a very different approach to how its technology orients digital content in physical space.

Tango is basically a smartphone/tablet with an AR enhanced camera. The camera combines three computer vision technologies that allow it to track and map its physical surroundings with incredibly high degrees of accuracy: motion tracking, area learning, and depth perception. Unlike image or GPS-based AR technologies, Tango uses this computer vision technology to generate a 3-dimensional map of the user's surroundings. This "map" is called an ADF (Area Description File), and subsequent Tango applications can draw upon the spatial data within this file to create additional overlays.

In many ways, Tango extends the logic of other Google mapping projects, such as Google Street View, just on a much more precise scale. One of Google's core missions is to "organize the world's information," but it seems like Project Tango is more of a pursuit to "organize the world" itself. What we're seeing with Tango (unsurprisingly) is a continuation of the digital hegemony that Google currently wields within screen-based media: they're not seeking control of AR content so much as the spaces and processes through which people will access it.

But who cares? Wouldn't something like project tango just move as that much closer to the era of "ubiquitous computing?"

Well, it matters because there are social, cultural, and political implications to technology companies tracking what counts (and what doesn't count) as potentially augment-able space. Take for instance Microsoft's promo videos for the Kinect. Their "retail clothing scenario" video depicts a customer walking into a clothing store where an AR mirror automatically overlays a digital dress. The customer does not do anything to approve of the augmentation nor the gendered clothing items the mirror (super)imposes onto the customer's body.

This kind of "proxy augmentation," in which digital overlays are subordinated to the physical objects they represent, is one of the most prevalent depictions of AR for commercial purposes. One of the leading partners of Project Tango, Lowe's, has invested in an AR app that overlays furniture within customer's homes so that they can see an accurate spatial representation of a new kitchen stool before clicking "Buy."

The trope of celestial augmentation merely serves to obscure this underlying capitalist impulse currently driving the commercial AR industry. Similar to any new technology (like a spaceship, for instance), we want to believe that AR will be leveraged for altruistic purposes and allow us to test the limits of human knowledge, wonder, curiosity, exploration, etc. When, in reality, it's far more likely to be leveraged for more mundane consumer activities, the same ones that have always colonized the digital and physical spaces of everyday life.