The Future of Social AR: 11 Speculations about Proximity, Truth, and Art
This post is an adaptation of my submission to the CSCW 2020 workshop titled The Future of Social AR, where I was a panelist among academic and industry experts in the collaborative AR space. The workshop brief was as follows:
As augmented reality (AR) technologies become more pervasive, there is a growing interest in Social AR systems designed to support face to face interactions with collocated others. Prior research on AR often focuses on the technical aspects of the technology. However, at this early juncture, it is crucial to reflect and discuss the ethical, political, societal, and privacy implications of Social AR. This workshop aims to bring together industry practitioners and academic researchers to discuss the opportunities and challenges of social AR: from platforms to content creation, to self-representation. We aim to use a design fiction approach where workshop participants create speculative scenarios that interrogate the values imbued into Social AR. Based on these discussions, we will put together a set of initial recommendations for designers of Social AR technologies.
In response, I proposed a collection of 11 speculative topics for critical design and dialogue around social futures in augmented reality. In the article below, I expand and elaborate on each topic through a generative discussion that foregrounds ethical, social, and cultural concerns.
Introduction
Vision
The future of social augmented reality is at once fantastic, dystopian, unsettling, and imminent. The public, intimate, political, social, physical, and imaginative spaces we inhabit provide a rich set of dimensions in which augmented social experiences can take form.
I envision social AR futures that radically reconfigure traditional proxemics, redefine social identities, and realize new media for collaborative art.
Background
My name is Janaki Vivrekar, and I am an EECS graduate student studying HCI and new media at the UC Berkeley Hybrid Ecologies lab advised by Eric Paulos. I’m intrigued by how emerging technologies can unsettle our existing social realities by creating avenues for novel interactions, co-creation of art, and new understandings about identity.
By participating in the Social AR Workshop, I aim to bring a critical design perspective to the conversation and introduce novel provocations in the social AR space. I am eager to ideate with and learn from others at the forefront of building the augmented social spaces of the future.
1 | Virtual "Unrealities"
Human touch simulation technologies will ambiguate physical and virtual interactions.
I envision a future where vibrotactile haptic feedback technologies will enable AR to blur the lines between physical and and non-physical AR human interactions. I imagine that AR devices of the future will unsettle the way we interpret “real” interactions, shrinking the the gap between virtual and physical touch perception.
Current research [2, 7, 8, 9] focuses on making virtual interactions seem more realistic, but how might we think about making real, physical interactions seem more virtual? For example, improved object detection in AR can help monitor human interactions with real people or physical objects in their vicinity and deliver specific haptic feedback to confound the user’s interpretation of those true physical interactions.
When users start interacting with real people or physical objects with haptic feedback provided by an AR system, then perhaps their tactile experience of virtual objects in an AR space will feel closer to reality.
This poses an opportunity to think about how we might transcend unreal virtualities (AR/VR experiences that don’t feel realistic) by designing virtual “unrealities” (AR/VR experiences that make the physical world feel unrealistic).
2 | AR Profiles
We will need to define and explore the AR Identity Space.
We bring our whole selves to physical interactions, and we use profiles as a proxy for our identities on web-based social platforms. What does the identity space look like in augmented reality?
As social AR experiences become more prolific in the future, I see a need to build interfaces, interactions, and experiences to help people communicate their identities to others who share their virtual or augmented environments.
Web-based social platforms like Twitter, Instagram, YouTube, and Facebook rely on structured textual and image-based descriptors of identity, an idea that Hollan & Stornetta described as “computing personals” in Beyond Being There [4].
The design space for AR Profiles is much richer, with the possibility of answering “Who Am I” questions with virtual instantiations of physical artifacts or AR playbacks of memories that are core to a user’s identity. Additionally, these virtual instantiations of identity-telling artifacts need not be rooted in the visual realm; for example, olfactory or auditory augmentation of reality could be compelling ways to communicate identity to others in a shared environment.
3 | Personal Publics
Social AR will produce a new level of "closeness" and raise questions of safety in personal publics.
Over a decade ago, many Twitter users viewed the platform as “personal media” [6], where they had “a large degree of control over what and how they communicate” [11]. Schmidt describes “personal publics” as spaces where information is curated according to personal relevance rather than “journalistic news factors,” addressed to explicit known network ties rather than an “unknown mass audience,” and transferred in conversational modes rather than publishing modes.
The intrusive and unchecked nature of disinformation coupled with manipulative, algorithmically-engineered walls of content on social media platforms like Twitter makes them increasingly inhospitable for personal publics to exist organically, safely, and ethically. How might social AR technology create safe spaces for people to build personal publics? For example, could social AR introduce ways to virtually “block” people from physical space, or blur out the faces of individuals who are intruding on safe augmented spaces?
Conceptualizing social AR experiences as spaces for personal publics to emerge also allows us to think about how we might physicalize a personal chat room, or bring direct messages to life in AR. I imagine a future where social AR experiences will introduce a new category of social “closeness,” somewhere between online acquaintances and close friends and family. This category could include someone I might invite into a personal virtual or augmented space but maybe not meet in person. How might we update common metrics like “tie strength” to account for new levels of closeness?
4 | Anonymous Reality
AR will enable and normalize anonymity in shared spaces.
As surveillance capitalism deepens its roots in modern social technologies, I am curious about how AR will create new ways for to preserve anonymity in shared spaces.
For example, I imagine AR facilitating social therapy support groups where individuals can engage with others virtually in comfortable, intimate settings and reap the benefits of vulnerable, reformative interactions without revealing the details of their true identity.
However, concealment of identity via AR opens up a large design noir space, where we might consider extremes like AR creating new environments for radical, extremist, and hate groups to recruit participants and provide augmented training in physical violence.
Web-based social platforms like Twitter, Instagram, and Facebook fail to prevent bad actors from taking advantage of anonymity to target, harass, manipulate, and exclude others (especially individuals with marginalized minority alignment). How will social AR technologists strictly prevent harassment and virtual spread of disinformation like most social platforms today?
Furthermore, I speculate that AR will enable violence toward vulnerable populations by objectifying [12] and harming certain human body forms and features in virtual spaces. How will we define legislation around criminal activity and crimes committed toward virtual human forms?
5 | Influence-ARs
A new genre of social media influencers will emerge in social AR space.
Social media influencers and content creators emerge on almost every social platform that becomes widely available and adopted by the general public. I envision a future where the same will be true of social AR platforms, as AR technology becomes more widespread.
Just as the “blog” and “vlog” emerged as ways of record-keeping, storytelling, and public sharing in textual and video-based digital media, I imagine a new type of extended “AR log” transpiring through increased public adoption of AR technology in their daily lives.
AR platforms will create a new space for social media creators to share content about lifestyle, interior decoration, travel, and more in augmented space. Artists who create in three-dimensional spaces will be able to share their process and outputs in a richer way, without having to collapse their results into a two-dimensional digital portfolio. Influencers will have new ways to market clothing lines, by providing augmented retail experiences for their target audiences [10]. Services like Skillshare and Masterclass will monetize exclusive AR learning options, where augmented reality will virtually bring experts into their users’ personal augmented environments.
Ethical concerns about influencer culture entering the augmented space include adverse impacts on vulnerable and impressionable populations that are susceptible to manipulation and could be subject to harm (ex: exacerbating negative beauty and body-image narratives targeting young teenage girls.)
6 | Ceci n'est pas AR
AR will be used to make collaborative graffiti, poetry, and surreal art in public spaces.
Collaborative art is a broad creative genre with communicational capacities that are largely untapped by mainstream social media. Augmented interactions in public spaces pose a unique opportunity for individuals to co-create persistent virtual realities in augmented space, even as the physical world erodes in different ways.
Extending accessible social AR experiences to the public will create new dialogues between physical destruction and digital permanence. People will find ways to digitally deface functional physical public spaces and repurpose dilapidated physical structures into augmented oases. Augmented realities will perplex scale, texture, location, and meaning of physical objects, creating new flairs of surrealism. People will embed poetries in augmented landscapes, rooted in but without leaving a trace on the physical world. Augmented amusement parks will become a new attraction.
Architecture delimits physical spaces, and buildings are artifacts of the physical world; while these constraints define the physical parameters of AR experiences too, what infrastructures will define and enliven the virtual parts of these experiences? Who decides what virtual infrastructures are built where? I envision a future where the tools for creation in AR spaces belong to everyone, where everyone can collaboratively engage in producing art that brings augmented spaces to life and create virtual identities for spaces that may even be distinct from their physical identities.
7 | Replay-R
Social AR will spark a new genre of locative data collection, caching, and sharing.
AR geocaching introduced a new concept of mobile-accessible and manipulable data that exists co-dependently in digital and physical spaces. Gamifications of this interaction modality (like Pokémon Go) are canonical examples of the lure of virtual nuggets embedded in physical space.
I envision a future where individuals will be able to create AR recordings or “replays” embedded in physical space, and share these with their social networks. Their social contacts could visit the same locations and engage with these replays asynchronously but feel as if they are occurring “live”. The locational caching of memories may become a new way of not only socializing but also historical record-keeping. How might we also learn from past successes and failures (like omissions and mischaracterizations by the New Philadelphia AR app [13]) of AR for historical preservation?
“Geography is not, however, secure and unwavering,” as Katherine McKittrick states in her seminal book Demonic Grounds [5]. We produce space and meaning in a Lefebvrian sense. Embedding augmented experiences relative to locations can contribute to the process of geographic domination and exclusion of certain groups. When AR replays of the past start inhabiting a particular physical space, the involved virtual bodies write the narratives of the present. In McKittrick’s words, “If who we see is tied up with where we see through truthful, commonsensical narratives, then the placement of subaltern bodies deceptively hardens spatial binaries, in turn suggesting that some bodies belong, some bodies do not belong, and some bodies are out of place” [5].
8 | Augmented Nostalgia
AR will present relics of the past as nostalgic experiences, in an increasingly dystopian world.
In the San Francisco Bay Area, the COVID-19 pandemic quarantine coupled with social unrest and dense smoke from California wildfires has given residents a taste of what the imagined apocalyptic state of the world feels like. Public spaces full of strangers are dangerous rather than recreational, and free, casual face-to-face interactions feel like a rare relic of the past. Instead, the “new normal” includes deserted public parks and strictly intentional, anticipated planned online interactions.
Socialization during and post-pandemic has warped the ways we engage with our friends as well as strangers. How is the pandemic introducing new social practices around interacting with strangers? How are these emerging practices affecting people’s characterization and recognition of the “familiar stranger”? What social practices of the past are no longer practical but still evoke nostalgia?
I envision AR technologies of the future producing ways to reimagine the function of public spaces as settings for immersive, nostalgic experiences that are at once personal and public. For example, imagine standing alone in an open space and augmenting the setting with a crowd of familiar strangers all around you. AR poses an opportunity to simultaneously experience being alone and being surrounded by strangers in the virtual world. Similarly, we might imagine AR technologies as tools for capturing and recording memories of the past.
9 | Protect-AR
We will use AR for physical safety in public spaces.
I envision a future where personal protective equipment (PPE) will be integrated in AR headsets, where AR interactions will begin to replace traditional physical interaction in public settings, and where AR devices will have physical safety plugins to make users aware of their own contexts.
As wearable AR headsets become more common in public, AR amusement parks and movie theaters will become more prolific, restaurants will serve menus in AR [1], and AR will start to form an additional layer between human-to-human interaction. One social benefit of these type of interactions will be to limit contamination and germ-spreading in post-pandemic contexts.
I also imagine AR physical safety plugins that filter reality through the lens of safety to help users navigate public spaces safely. For example, AR experiences could detect how densely crowded a space is and how many people are wearing masks, and display a colored overlay or virtual floating germ particles to indicate how risky it is to be in the space.
Such AR technology could even indicate “safe spaces” for marginalized groups. However, using physical feature detection to determine which spaces are safe would create ethical issues; for example, imagine a genre of racist, sexist, and violent AR applications that physically exclude certain groups of people from public spaces, by equipping dominant groups with augmented perception of their surroundings.
10 | Counterfunctional AR
AR can be designed to limit its own functionality.
Employing functional oppositions in the design of digital tools can create “strangely bizarre”, “strangely problematic”, and “strangely useful” experiences [3]. Current AR tools rely on replicating familiar patterns of the physical world to produce convincing and comfortable virtually augmented experiences. However, there is a provocative opportunity to build deliberate unfamiliarity into AR for bizarre interactions with the physical world, or serve AR experiences that confound users and make them question their expectations from augmented reality.
How might we design AR interfaces that obfuscate, complicate, or render unusable certain elements of augmented experiences? As VR/AR devices are just starting to become more widely available and accessible to the general public, how might we lean into the intrigue of AR/VR to create unexpected experiences?
Additionally, how might we use augmented reality to complicate traditional interactions with existing digital interfaces like mobile and laptop screens, or counter common design patterns like intrusive notifications?
11 | Beyond Ocularcentrism
Sensory AR experiences may sometimes exclude visual immersion entirely.
The current AR market and research space almost always includes visual displays or immersive visuals as foundational elements of the experience. While tactile, auditory, olfactory, and limited gustatory sensory elements have been explored, they are often seen* as supplementary effects or inputs to the dominant visual immersion that characterizes “augmentation” of our reality.
How might we move away from the ocularcentric approach to augmenting reality? In what ways might we center other sensory inputs and outputs in AR experiences? How might sometimes excluding visual displays from AR open up space to innovate and explore other multisensory combinations? If we imagine our elementary senses as a 5-dimensional basis, how might we explore other “linear combinations” to produce novel experiences that might even produce extrasensory effects?
Ocularcentrism is pervasive across all facets of communication and society. For a local, in-text example, I called my future-looking statement on page 2 a “Vision”, and I might refer to thoughts about the past as a “Reflection” — words tied heavily to my sense of sight. Imagine reconfiguring our models of temporally situated thoughts in parallel to our other senses. What if I were to instead call my “Vision” a “Scent” or “Pulse” or “Flavor”, or refer to “Reflections” as “Echoes”?
As absurd as these linguistic suggestions might be, I believe there is value in pursuing a similar line of questioning for our definitions of augmented reality, and whether vision truly is central.
* Yet another example of how ocularcentrism is ingrained in our language systems.
Conclusion
In this document, I shared 11 novel critical design discussions about social AR futures. Within the topics of discussion, I described opportunities for immediate innovation, incremental design progress, and far-reaching speculation. In the Social AR workshop setting, I plan to extract and share a few chosen topics framed as critical design prompts, presented in a visual and engaging format.
The future of social AR is lush with opportunities to reimagine human-to-human interaction in the physical and virtual dimensions, produce new representations of social identities, and engage new forms and spaces of collaborative art. From my vantage point as a growing scholar in HCI and new media, AR is poised to simultaneously situate and unsettle our social futures.
References
[1] Alkanoids. 2020. Augmented reality (AR) is the future of Restaurant Menu? https://arvrjourney.com/augmented-reality-ar-is-the-future-of-restaurant-menu-4c66704b1142
[2] Inrak Choi, Heather Culbertson, Mark Miller, Alex Olwal, and Sean Follmer. 2017. Grabity: A Wearable Haptic Interface for Simulating Weight and Grasping in Virtual Reality. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST ‘17). ACM, New York, NY, USA. DOI: https://doi.org/10.1145/3126594.3126599
[3] James Pierce and Eric Paulos. 2014. Counterfunctional things: exploring possibilities in designing digital limitations. In Proceedings of the 2014 conference on Designing interactive systems (DIS '14). Association for Computing Machinery, New York, NY, USA, 375–384. DOI:https://doi.org/10.1145/2598510.2598522
[4] Jim Hollan and Scott Stornetta. 1992. Beyond being there. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '92). Association for Computing Machinery, New York, NY, USA, 119–125. DOI:https://doi.org/10.1145/142750.142769
[5] Katherine McKittrick. 2006. Demonic Grounds: Black Women and the Cartographies of Struggle. http://www.jstor.org/stable/10.5749/j.ctttv711.
[6] Lüders, M. 2008. Conceptualizing personal media. New Media & Society, 10(5), 683–702.
[7] Lung-Pan Cheng, Eyal Ofek, Christian Holz, Hrvoje Benko, and Andrew D. Wilson. 2017. Sparse Haptic Proxy: Touch Feedback in Virtual Environments Using a General Passive Prop. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). Association for Computing Machinery, New York, NY, USA, 3718–3728. DOI:https://doi.org/10.1145/3025453.3025753
[8] Pedro Lopes, Alexandra Ion, and Patrick Baudisch. 2015. Impacto: Simulating Physical Impact by Combining Tactile Stimulation with Electrical Muscle Stimulation. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15). Association for Computing Machinery, New York, NY, USA, 11–19. DOI:https://doi.org/10.1145/2807442.2807443
[9] Pedro Lopes, Sijing You, Lung-Pan Cheng, Sebastian Marwecki, and Patrick Baudisch. 2017. Providing Haptics to Walls & Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). Association for Computing Machinery, New York, NY, USA, 1471–1482. DOI:https://doi.org/10.1145/3025453.3025600
[10] Retail clothing try-on via mobile and kiosk augmented reality. 2020. From https://www.youtube.com/watch?v=gZkMHygfwJ0
[11] Schmidt, J.H. 2014. Twitter and the rise of personal publics. Twitter and society, 3-14.
[12] 2019. Augmented Reality Will Objectify Women. The More Accurate Guide to the Future, timeguide.wordpress.com/2019/03/16/augmented-reality-will-objectify-women-2/.
[13] 2015. New Philadelphia Augmented Reality Tour. From https://www.nps.gov/subjects/ugrr/community/news_details.htm?NEWSID=4933850
