Abstract
Augmented Reality (AR) has existed for over five decades and has traditionally been explored as a medium for delivering single-purpose applications, often for entertainment. More recently, however, there has been a growing interest in integrating AR into users’ everyday lives. This shift has been driven by significant advancements in the development of modern AR devices. These devices are no longer handheld or obtrusive; instead, they resemble regular spectacles or fashionable sunglasses. Not only are they more discreet, but they are also paired with services that provide social and practical benefits through the use of always-on cameras and other sensors.
We anticipate that, in the future, almost everyone will wear AR devices that deliver tailored information to each user. This technology, termed "Pervasive AR", is defined as always-on, continuous systems that are aware of and are responsive to users’ contexts. When users are constantly exposed to a system that anticipates their needs and delivers tailored information on the go, their behaviour may change as a result. Thus, it is safe to assume that such an omnipresent system, integrated with every aspect of a user’s life, will present ethical and social implications.
Pervasive AR systems have already been examined in terms of the threats they pose, with the majority of the work focusing on the privacy concerns arising from their use. Pervasive AR has the ability to continuously record a user’s surroundings, thereby subjecting bystanders to constant sousveillance, which raises concerns related to both privacy and data security. Thus, exhaustive research into the privacy implications of Pervasive AR focuses on its effects on bystanders, technical implications, and various technical options to improve privacy.
Additionally, research has also investigated the social acceptability of these systems. This work has mainly focused on the form factor of the device and its acceptability in social contexts, as well as the effectiveness of various input gestures. However, it is essential to recognise that the context-aware and continuous nature of the technology, along with its ability to remain unnoticed, can lead to changes in social behaviour and impact both users and non-users alike. These investigations should be conducted empirically, drawing on the insights of potential users of Pervasive AR systems.
In this work, we explore the potential ethical and social implications of Pervasive AR by exposing participants to purpose-developed technology probes, providing foundational insight into the responsible design of such systems. A technology probe illustrates the functional use of a specific technology—in our case, Pervasive AR. Each probe simulated a distinct context of use, allowing participants to experience Pervasive AR first-hand. This approach was mentally less demanding on participants and enabled them to share informed reflections on how Pervasive AR might influence their social interactions and behaviours, while also fostering awareness of this emerging technological shift.
In this thesis, we demonstrate that the everyday use of Pervasive AR gives rise to several social and ethical challenges. Based on a guiding framework intended for emerging technologies such as Pervasive AR, we identify implications that affect individuals and society as a whole. Among the discovered challenges, the most recurrent and crucial issue is the asymmetry that Pervasive AR systems create, as they deliver tailored information to each user based on the particular user’s context at a given moment. The asymmetry was not limited to the tailoring of information but also extended to who had access to Pervasive AR.
Moreover, we discuss the importance of consent when using Pervasive AR systems. We identified that a sense of control was crucial in terms of what information is shared, what information is consumed, and the nature of the information being consumed. Our first study examines the use of Pervasive AR systems in shared public settings, revealing that they can lead to dyadic distrust among users and the presented information, due to the asymmetry the system creates. Distrust rooted in asymmetry will negatively contribute to social interactions, leading users of Pervasive AR to feel isolated and manipulated.
Our second study investigates the influence of face recognition via Pervasive AR on users’ social behaviours. As the face recognition-based information about a user’s bystanders was tailored to the user’s requirements, an asymmetry was created in the levels of information each user received about others. This resulting asymmetry leads to a sense of power imbalance among users in social interactions. Additionally, the integration of face recognition results in a significantly reduced sense of control for users in terms of their self-presentations and the formation of first impressions. These information-based first impressions precede actual interactions, urging users to make social decisions solely based on the information presented.
We expanded our exploration of integrating face recognition technology with Pervasive AR by conducting an in-situ, multi-participant empirical study, in which participants received role-based, scraped information about others via face recognition. In this study, we not only focused on users but also examined the social and behavioural changes among non-users who were indirectly exposed to this technology. Confirming several of our previous findings, we observed that non-consensual information scraping and role-based access raise significant ethical concerns. In particular, role-based access exacerbates existing asymmetries, leading Pervasive AR users to marginalise non-users.
Finally, our last study was driven by participants’ curiosity about the role of eye visibility in social interactions through AR glasses. This dialogue is particularly relevant with the rise of modern AR glasses featuring opaque lenses—such as the Snap Spectacles, which were used in most of our studies—raising the question of whether future AR glasses should resemble sunglasses with opaque lenses or transparent, everyday spectacles. Accordingly, we conducted an exploratory study that evaluated various AR devices, allowing participants to experience different levels of eye visibility through a Snap Spectacles–based technology probe. The findings highlight that eye visibility is crucial to the social acceptability of AR, suggesting that future AR glasses should be designed to preserve, rather than obscure, users’ eyes.
It is evident from our findings that Pervasive AR pose substantial social and ethical concerns. These implications warrant careful consideration in the design of rapidly emerging Pervasive AR systems. Among these, information equity and consent emerge as critical attributes for designers to prioritise in order to mitigate issues arising from asymmetry and a lack of control for both users and non-users. Furthermore, it is crucial that the design of Pervasive AR is conducted in collaboration with its potential stakeholders through ongoing civic dialogue, rather than in isolation. Our findings create a space for proactively addressing these stakeholder concerns related to Pervasive AR. Therefore, we anticipate that our work will serve as a starting point for a responsible and ethical development of future Pervasive AR systems.