The Design Observer Group


Posted 06.09.10


Mitchell Schwarzer

A Sense of Place, A World of Augmented Reality: Part 2


[Credit: William Hook via Flickr]

Palm Readers
Had Nino Badalamenti been taken on his Mafioso drive through New York in 2010, and had he been permitted to use an iPhone along the way, it’s likely that his outing — discounting the ominous presence of the gangsters — would be altogether different. Applications would beam detailed information about the place he moved through. He would find his location on a mobile GPS map, track the route and access streaming historical information or commercial suggestions from passing sites. He’d photograph the slivers of skyscrapers, and perhaps send Tweets to his online network. His ever-present connection to the Web would counteract any sense of displacement. The perceptual rhythms, oscillating back and forth between direct sights and websites, would, however, interrupt the flow of experience, flattening his flights of imagination, merging his impressions with the insights of others, structuring his sight within the parameters of application programs. He’d be lost in unfamiliar geography and found through information overlays.

A host of handheld devices and applications now promise to endow individual perception with the experience of a long-time local and the knowledge of an educated specialist, combining aspects of the depth of place Nino acquired during his Sicilian childhood and youth as well as the breadth of place by which he operated as automotive professional in Milan. It all began in 1999, when the BlackBerry pioneered wireless email. That same year, NTT DoCoMo offered access to the Internet on a cell phone for the first time. Subsequent developments have been dizzying, catalyzed, around 2003, by a synergy of high-speed wireless Internet connectivity, commercialized satellite photography and the evolution of the digital handheld. Since then we’ve witnessed the onset of Social Networking Sites (SNSs) like MySpace and Facebook; image and video sharing sites from Flickr to YouTube; micro-blogging from the field, epitomized by Twitter; digital mapping technologies on Google Earth and Yahoo Placemaker; and the handheld PDA and cell phone merged into a miniature, portable computer.

Two developments stand out with respect to our sense of place. First, as real world activities migrate online, consumers are turning into producer/consumers. The kind of top-down, high or mass culture epitomized by the museum, newspaper or television is dissolving amid a plethora of platforms for communication and entertainment, many of which are generated by individuals and their online communities. Second, in any given locale, the network is augmenting reality. In our pocket we can carry anywhere an encyclopedic library and archive, a knowledge-screen that can be superimposed on our visual field to deepen its appearance along the vectors of our shifting inclinations.

A Lot of Help from My Friends
Social networking sites transform the endless reaches of the Web into select familiarized destinations — the sites of oneself and one’s friends. [1] The attention given to these online contexts appears to transport us away from direct, place-based encounters. Instead of a full-sensory conversation occurring in real time and accompanying an activity someplace, we click to someone’s web page to view the digital composite of a person — a resume/photo album/personal statement/daily itinerary — splayed across a screen. In such distanced, asynchronous communication, the physical nuances of bodily gesture are lost. So too are social constraints on behavior. People in pre-modern society were born into fixed roles. Modern society offered the chance to perform different roles, given one’s ability to negotiate the terrain of new businesses and institutions. On the Internet, where we can adopt deviant roles, the roots of role-playing in place appear to have been severed.


Facebook Fan Page.


As researcher Danah Boyd describes these arenas, where digital flaneurs flock to see and be seen, social networking sites do not so much map real world society as inaugurate a new way to be social. [2] That new way constitutes a re-scrambled iteration of community/society: neither the embedded, blood-and-soil village dweller of pre-modern days nor the liberated modern flaneur floating through a world of enticements presented by anonymous others. Social networking sites foster the intimacy of community but with minimal obligation, the choices of society ameliorated by an in-your-pocket means of recognition.

This apparently inward turn, away from direct experience, is affecting what we see and do in real places. On social networking sites, people view images produced mainly by their friends, even if many are mash-ups of mass content and individual creation. Daily, users are treated to slideshows of others’ adventures and exploits. The collage-nature of those shows dissolves the line between monument and ordinary landscape, since personalized comments and narratives can enliven someone’s bedroom into as vivid a scene as the Empire State Building. That’s not to say people aren’t encouraged to go to Manhattan and explore (and digitally capture) the skyscraper. Representations of exploits in the real world enhance one’s own site. Indeed, the social pressure to build up a site’s visual power and reach may be leading some users to pay more attention to place. One must go out into the world and record its aspects in order to concoct a compelling digital display of one’s self. Individuality, within the highly produced graphic environs of social networking sites, depends on immersing oneself in the eye-catching reaches of society. Armed with a handheld device, each person becomes a diarist, reporter, photographer and videographer, relaying information and personalizing an online community’s experience of place. Thus social networking de-professionalizes our online viewing field, allowing us to wander in the gardens of visual spectacle created by amateurs like ourselves, even as it professionalizes our own engagement with place, encouraging us to mine for representational gold in untold previously overlooked locales.

Layering Reality
Not only are online communities mixing enticing cocktails of person and place; we’re also taking the Internet with us on our excursions, and, along the way, further blurring the lines between direct, mediated and virtual experience. In 2009, a convergence of cell phone display technology, geographic tracking devices and computer software ushered Augmented Reality from the research laboratory into the palms of millions of cell phone users. Augmented Reality refers to an overlay of the Internet atop one’s perceptual field. It is facilitated by handheld displays that mediate the view of a place through a camera lens and augment that view with virtual information. Cameras register surroundings, which are then analyzed by image recognition software/algorithms; GPS antennas locate a site’s coordinates, and internal compasses' motion sensors pinpoint viewing angles. Places are geotagged or located on the network, and then bundled with information.


Layar,  a mobile Augmented Reality app.


Like spiking a drink, Augmented Reality punches up both our visual field and our consciousness. Sight lines travel through light, shadow and sheets of information. Displayed on the viewfinder, these portals from place to network sometimes sharpen what’s visible and other times extrapolate on what’s not. Point your camera at a mountain range and pull up the names of the various peaks. Point it at a building and read its name, date of construction, and architect, not to mention scroll through photographs of its past. Applications spread a building’s future before one’s eyes, showing, if it’s for sale or rent, specifications, pricing, and interior views. Orient a device within a given commercial intersection and gather shopping, eating and entertainment options and ratings in the vicinity. Wikitude, a travel guide by Mobilzy, geotags the famous and not so renowned places of the world — over one million so far — and overlays Wikipedia sites atop them. Layar plasters real-world images on one’s cell phone with Twitter messages, classified ads, consumer reviews and incidental pictures from sites like Flickr. The Nearest Tube allows users to tilt or pivot their phone and display the locations of transit stops on the screen map before them. Latitude, a social networking function of Google Maps, allows users to hold up their handheld and locate their friends’ home pages and current activities with respect to geographic coordinates; you see Tweets and personal messages tied to specific locations.

The enhancement of reality is visual, cognitive and somewhat haptic. We’re not only looking at something, we’re not only thinking about it, we’re using our fingers to probe into the larger dimensions added onto it by the network. Hidden Park, a game by Bulpadok, turns an otherwise unremarkable public park or playground into a field of fantasy. Participating in contests, children search through bushes, under boulders and atop meadows for trolls and other apparitions, virtual characters that pop up whenever the handheld device is held close enough to them. “When you walk down the street it’s usually pretty boring," said Ori Inbar, founder of AR game maker Ogmento in New York City. [3] Real streets, or parks for that matter, can’t compete for a child’s attention with the high-voltage visceral entertainment available on television or video games. Inbar got involved in Augmented Reality so he could find a way to blend his kids’ love of the computer screen with their negligence of the outside world. Through such Augmented Reality games, children move through real space at the behest of virtual space, getting their kicks from those winning moments when they connect the two realms. By encouraging children to look more closely at real places, the games might lead to an aftereffect where they explore place outside the game. A pond in a park that once signaled the grotto of a genie might now cause them to wonder about the watery world beneath their feet, the actual hydraulic infrastructure of the city explained to them on a website. Imagine the computer leading us back to the city and nature.



Hidden Park, Augmented Reality children's game.


The chief contribution of Augmented Reality is its potential to take a place, anyplace, and enliven our perception of it with other visual, geographic and verbal representations. Applications launch a play about place in which we start out as audience and end up as director; a production that we tap into, contribute to and then convey back to the network. We approach, apprehend and adjust to the world around us by carrying another world with us. We stroll with a docent at our side, answering our every query. Everyday place is curated like museum space, the most banal streetscape brought to comparative life by earlier photographic iterations or enriched as to its unique aesthetics, demographics, economics or politics. Augmented, any reality takes on the trappings of a documentary film, a college lecture, or a police investigation. Augmented, overlooked stretches of city, suburb or countryside can field the kind of engagement that has long been the privilege of famous or infamous attractions. Augmented, all places become events.

Instead of moving through blank space between bright landmarks, Augmented Reality allows us to fill in the blanks. It makes palpable local environments that were largely shorn, during the modern era, of their utility and hence need for visual legibility and attentiveness. Yet what will augmented places be like? Will they resemble pre-modern place? Can the aura of place be rediscovered? Sure, we’ll gain rapidly a version of the embedded knowledge of place local people gleaned over a lifetime, but we’ll also apprehend that knowledge as a spectator or temporary user, a tourist gliding through. Ideas and images will pile up on the screen, and the pile up might contribute to our impatience, might feed our hunger for other fleeting engagements, our lack of commitment — the click, skip and scroll of contemporary perception.

Like the price quotes for a hotel that layer atop one another when we undertake an Internet search, information will come at us from a variety of far-flung and out-of-control sources. Because of our growing networks of friends on social networking sites, some of it will be personalized. We’ll be walking through places where our friends have walked (or digitally trolled through) and their messages, like leaves in the wind, will blow into our path. It will be comforting to journey with an individualized/community perspective. Yet when we’re in a remote part of the world, will we want our “friend’s version” to resound? Is escape from our online community possible? And how will we discover things ourselves with so much assistance and familiarization?


Wikitude, a mobile Augmented Reality app.

No doubt the sheets of information stacking in our visual field will often be as anonymous and loud as any current of communication in modern times. Will the ambient features of a place be even more tuned out? Since we’ll spend so much of our time connecting direct and virtual realms, the view of a scene before our eyes with the map or info-pops on the screen, circumstantial sensations will be shunted aside or unobserved. Select augmented aspects of a place, say, the status of a building within some category of architectural worth or historical importance, or the rating of a restaurant's wine list or noise level, will dominate experience. Names, categories and figures will predominate among the kind of data through which we apprehend place. Zooming into a named place on our screens, we’ll probably zoom past most of that which lies astride and around it, and as likely zoom away to other distant sites before coming back to where we are — if we ever do return. Just as many teenagers today prefer watching highlight reels from sports contests instead of the entire game, why should we think people using Augmented Reality will have the patience to wait for a place to reveal itself over a long time span? Won’t they too hang on the highlights?

Rather than fostering a shared and lasting visual consciousness with regard to place, the information conveyed by Augmented Reality reflects the centrifugal and commercially spellbinding tenor of the 21st century. To catch our fleeting attention, content providers will doubtless swell the facets of a place to shrill and bawdy levels. We will truly come to know the skeletons in a building. Likewise, since the screen interface will clutter with banners and pop-ups, we’ll be lost in clouds of commerce and communication. Places will be ranked, overly conceptualized. Marketers will follow where we click and our geographic voyages, which we feel are liberating, will become one more structure for attaching us to consumption. We’ll be driving down a street someday and not only will advertising pop up on our individual screen, but billboards too might be synchronized to change marketing options based on a passing motorist’s history of shopping and browsing. Why wouldn’t the marketplace infiltrate Augmented Reality to the extent it has every single technology — think newspaper or radio or television — so far? We think we’re opening a portal into the deeper reality of a place. We’re actually developing our marketing profile. No doubt, versions of the billboards and signage that obscured the roadside or the commercials that tore apart television programming will provoke some of us to click away from the virtual visual chaos lying in the palm of our hands.

Will we be diverted more than ever from happenstance? While we’ll probe along the tracks of our individual interests in architecture, cuisine, entertainment or shopping, we’ll probe really deeply only when we’re motivated to produce some images and verbal impressions, contributions to transmit back to the network — our online community. And then, our observations will be calibrated to the network’s conventions. We’ll have to deal with Google Maps or Wikipedia or TripAdvisor, with their rules and systems. The more such sites coordinate our experience of place, the more they will make up its infrastructure.


Google Maps, the White House, Washington, D.C.

There will be less need for individual memory. The network will remember for us. Half a millennium ago the transition from oral to written culture brought about the dominance of sight over hearing. “The printed page,” writes Donald Lowe, “with its standardized type, punctuation, and sectional divisions, gradually accustomed the eye to the presentation of messages in a formal, visual space.” [4 ] Augmented reality will likely accelerate that process of relocating knowledge to a shared, objective realm. Its ability to call up knowledge of the world — putting names and ideas together with buildings, landscape features, even people’s faces — will mean our minds will be freed up from such mundane tasks. Why memorize the details of any given spot or style? It will be more advantageous to acquire the facility for expertly negotiating the network, moving with agility across the digital/real landscape. There will be less reason to bore into things and more reason to look askance. We will encounter family, friends and colleagues and, having an online version of them readily available, be prone for less storytelling and more debate. Critique and measurement will grow. Our minds will be freed to compartmentalize — where we negotiate different place attachments or needs at the same time — or access contingencies — where we compare multiple sensorial fields. Multi-tasking of the future will dwarf anything we’ve yet experienced. It’s unlikely that modern society’s tendency to drift from place to place will be arrested. Place augmented will become place further collaged and remixed.

Going back and forth between direct and augmented perception, we might lose track of their distinctions. Modern society’s nomadic existence amid vast deserts and select green oases — billions of strangers, thousands of celebrities, and perhaps hundreds of familiarities — might soon find itself in an endless jungle of ideas and images where perspectives on what’s natural and what’s artificial are impossible to come by. Social Networking and Augmented Reality will pull place into themselves, into the network as us. One can imagine Nino Badalamenti being driven around Manhattan no longer looking at the street-side scene, but, instead, holding his camera out in front of his face like a spyglass, while watching a clip of himself in the very same spot from Mafioso and then clicking to another episode.

New Epoch 
In wearing all of mankind on our skins (or in our pockets and palms), it seems to me that we are moving into nothing less than a new epoch of social relations — when we will be neither the community member embedded in his or her tribe and place nor the individual floating or flowing in placeless society produced by omnipresent yet unapproachable elites. Via Social Networking and Augmented Reality, we are finding options for gaining and communicating knowledge in any given place expanded beyond the constraints of olden and modern times. We always know where we are and what we’re seeing. We are somewhat aware of who is assisting us and how they intend for us to experience, value and own, to varying extents, the place at hand.

Up until now place has been the smallest unit of landscape and architectural observation, keyed to an immersion over time in the vagaries and particularities of site. Place stood at one side of a geographic spectrum whose other end was the global or, alternatively, the universal. In modern times, the particularity of place was recalibrated along the widespread, speedy and abstractly visual reaches of mechanized vehicles and imagery. The global supplemented the local and often shone alongside it. Pride of place could turn nostalgic and against the machined. Nowadays, the portable network handheld suffuses place with the global, promising it renewed vitality and transforming it all the more, turning what’s singular into the shaped, contrasted and ranked, sprinkling knowledge and entropy across the land.