Mixed Reality (or for short, MR) has transformative potential, especially for modern work and collaboration. We use the term MR to encompass a variety of immersive technologies, differing in terms of device form factor and the user experience they create – ranging from devices that minimally augment reality (AR) to ones that fully immerse the user(s) into a virtual reality (VR). This continuum of MR experiences has a variety of applications, where AR is good for some things and VR for others, and we will cover multiple examples along this continuum.
Mixed Reality for the Future of Work
In the future of work, MR may very well play a crucial role in our productivity and creativity, and collaborative work with colleagues distributed across the world. We can imagine that MR will be available to us as an immersive alternative to our existing devices (desktops, laptops, phones, and smartwatches).
However, the ways in which this technology becomes available more broadly is still unclear. What device form factor will be sufficiently comfortable, capable, and socially acceptable at work? Which hardware and software technologies will be needed to make the breakthrough to adoption? How should users interact with MR content? What is the “killer app” for work?
These are some of the questions we will grabble with in this course, with the MR technologies available today. Today, MR is available as a range of devices that sit at different points on a continuum of user experiences, which we call the Reality-Virtuality continuum (or simply, the MR continuum).
The Mixed Reality Continuum
The continuum was introduced by Milgram & colleagues (1994) to capture the large variety of user experiences that exist between reality and virtuality.
The range of technologies along the continuum have different pros and cons, which support different use cases.
In this introductory lecture, we will explore a variety of example applications – with focus on the breadth of user experiences (next lecture, we worry about the technical implementations).
At the “VR end” of the continuum, we have fully virtual worlds. This is the kind of experience that has had most real-world impact today, with immersive entertainment experiences (such as gaming … or even a VR time machine). But it also has work-oriented use cases which has led to recent business cases, such as for professional training, or practicing presentations and high-stakes social interaction). Here, the design goal is to fully immerse the user into the experience so as to forget about their physical reality and adopt the illusion of a new virtual reality – and this illusion can be quite strong! These kinds of experiences are becoming increasingly easy to develop in the AI era, with generative models for world building, such as World Labs, Meta Creator Assistant, and XRBlocks+Gemini (XRBlocks will later be introduced in a guest lecture by the inventor, Ruofei Du from Google).
At the “AR end” of the MR continuum, the goal is different. It is about situating virtual content in the physical world. This has potential to become the next everyday interface for computing, enabling real-world browsing and navigation, gaming, and learning. But professional applications are starting to find their footing; incl. uses for advanced manufacturing, remote assistance, and immersive conference calling.
Throughout the course, we will cover, in much more detail, some of the use cases for work – along with the underlying technology and interaction paradigms necessary for creating user experiences on the MR continuum.
MR/AR/AV/VR/XR – I’m confused…?
Before we start, let’s get the terminology straight.
Speicher & colleagues (2019) conducted a survey + expert interviews to answer the question: What is Mixed Reality? And the takeaway is that there is no universally agreed upon definition. However, in our course we adhere to the definition visualised in this short explainer video (by the last author of the paper):
- Milgram’s MR continuum extends from Reality (R) to Virtual Reality (VR).
- Virtual Reality (VR) is when you are inside a fully virtual world.
- Augmented Reality (AR) is about adding content on top of the real world.
- Augmented Virtuality (AV) is the inverse of AR, where elements of the real world are added into the virtual world.
- Mixed Reality (MR) involves the merging of real and virtual worlds somewhere along the continuum, which includes AR and AV.
- Extended Reality (XR) is a fancy umbrella term that is very easy to say – and it sounds kind of cool … “XR”. The X is a wildcard for any of the above experiences.