I am not the first to ask a particular question, and I certainly won’t be the last. As we progress, the answer will shift. As more technical and societal paradigms evolve, it will shift again. So I’ll ask and answer in 2019, knowing full well it could be bupkis. But for now, humor me: What is Immersive?
Surprisingly, many definitions of the term shifted in the last 20 years to mean, “noting or relating to digital technology or images to actively engage one’s senses and may create an altered state” (Random House Unabridged Dictionary). When did technology come in, and what are these altered states? Taking a few steps backwards to identify the origin of the word, we can identify what is immersive, and what isn’t. This will be my last writing involving etymology, I promise.
Prior to 1990, immersive wasn’t popular in its adjectival form— it might’ve not even existed before the 19th century. The origin is the verb transitive “immerse,” which formed from the original Latin root immersus, meaning ‘to plunge’. Somewhere around the early 1600s, religious texts described baptism as the metaphorical connection between physical submersion of the senses and complete mental engagement or devotion to Christ. (This ‘complete mental engagement’ is also known today as Mihaly Csikszentmihalyi’s “FLOW”.)
This seems to be the turning point when immerse gains another definition: “to overwhelm; to involve; to engage deeply”, that finally went into print in early 1828. From this, we can derive a better-encompassing description of the adjectival form immersive: that which engages and involves.
Engagement & Involvement
For something to be immersive, it must have some degree of engagement and involvement, although these two similar qualities primarily differ by the amount of interactivity or agency afforded to a participant. Consider it a response spectrum: Engagement is the mental processing of an external world, and Involvement is the series of interactions with that world. Experiences can be weighted towards either side, but still holistically qualify as immersive.
Simply, one can be immersed in something at varying levels depending on how engaged or involved they are. An avid football fan jumping and cheering for their team on the television is equally engaged in an immersive construct of their mind as someone playing the original TETRIS. Neither the sports fan nor the TETRIS player are actually performing their represented actions (coaching football or assembling towers of blocks). They are watching these actions on a screen at a distance. Yet they both enter a different mental state, in which their attention is primarily on the screen as a stimuli source, anticipating and responding to new information.
Comparatively, the sports fan and the TETRIS player are immersed in different ways. The fan has no control over the plays of the game; he or she is given no affordance to change outcomes, yet watches hunched over with anticipation. They are mentally engrossed in the actions yet to unfold, akin to reveals in episodic television. While they certainly hope a play performed by their team goes well, they watch and mentally process the stakes. This type of immersion is primarily Engagement. Many of us can relate to physically stepping between someone so engulfed in a game or show. It gets nasty.
TETRIS, in its basic form, provides an aesthetic with an emphasis on gameplay. The primary function of the game is to act and react over time, and requires user input to move forward. Unlike American football, the game as a system of constraints is highly ergodic, allowing for the player to react in anticipatory play and build algorithms to prepare for an unknown: what will my next block be? There is no inherent story to TETRIS, nor is there gameplay beyond clearing horizontal levels. But these simple constraints allow a player to be mentally engrossed by the gameplay, as if in a trance. As a child, I had a caretaker that was very much this way with his old TETRIS handset in his LA-Z-BOY recliner. This interactivity and feedback creates immersion that is primarily Involved.
Before moving to examples, I want to clarify that these situations depend on the person. Someone completely engaged with football on television might disengage with TETRIS. They may not like the game. This rings true for those who love to read, perform crosswords, or play Sudoku. These activities, like all activities, have a range of immersivity and should never be considered hardset.
Punchdrunk International’s Sleep No More is very engaging; the actors fluidly perform through a variety of intricate sets that promote audience exploration and discovery. However I would argue that the show isn’t always involved, except in the rare “one-on-one” and pre/post-show spaces. For the majority of the performance, participants are spiritual observers with little to no agency over the events that unfold. The performance as a whole does not waver depending on the presence of the audience or lack thereof. Therefore it is up to the participants to craft the narrative in the space mentally, piecing together a story through spatial understanding and viewership of the performed vignettes.
This graph depends heavily on the individual— this one is mine. While the items are displayed as points, they are better represented as ranges of immersivity. (I doubt anyone is engaged like I am in Sudoku)
This does not imply that Sleep No More isn’t immersive— it is very much so, and highly recommended for anyone studying immersive theater and story-rich spaces. Rather, I consider it immersive in a predominantly engaging way. While participants piece together a narrative on their own, they rely primarily on their mental engagement. The observations of the characters’ dynamics, the clues hidden in drawers and shelves, and the rooms themselves are provided to aid in the detective-esque role. Physically searching for these clues, along with the fabled actor on-on-one moments, are oases of intense involvement for participants, and this contrast makes those moments more powerful to create an altered state.
Experiences that are more involvement-weighted usually stem from games and interactive entertainment. Titles such as Beat Saber, Rock Band, and Audica fit into this category. These experiences rely on systemic knowledge and procedural mastery that allow for physical immersion. Very few believe they have actual lightsabers, guns, or guitars; they are immersed via the processes and feedback of these items and their interaction with the worldspace. The story of these games is not external to be discovered, but built upon self-growth for the player, and mastery of a system. Tycho from Penny Arcade puts it plainly:
“I play games to enter a trance state and experience other lives, [Robert] plays them to defeat the designer of the game by proxy. That’s a significant distinction.”
Together, engagement and involvement define the immersivity of a world— based upon the combined placement of immersive senses upon a participant and the interaction techniques used to facilitate meaningful and impactful involvement. That’s where designers come in.
What is our goal as immersive designers?
Visual. Games. UX/UI. Interaction. Narrative. Worldbuilding. There are umpteen types of designers, far more than I can list. But the commonality between them is the desire to embrace human behavior, and craft meaning for it from disparate stimuli. The Immersive Designer aims to establish presence — a sense of “being” somewhere or someone/thing — using lessons learned from distinct designs of the past.
Presence, as Jeremy Bailenson describes it in Experience on Demand, is “the sine qua non of VR.’’ It is both a goal for designers to achieve while simultaneously a tool in our set to convey mechanics and narrative. Yet achieving presence alone is more than difficult; a key factor is the suspension of disbelief, a moving goalpost depending on a participant’s background and mental state prior to an experience. Bailenson elucidates further on the types of participants, categorizing them as high and low-presence individuals when they are brought into a virtual environment. “High-Presence” participants suspend disbelief sooner and with less sensory stimuli change required. A generated low-polygon world (or a game of make-believe) can be enough for these types of players to accept a mental state change. “Low-Presence” is the opposite: those individuals are less easily brought into a state of psychological presence, and higher fidelity stimuli might be required.
Some would call this the Peter Pan Principle. Carrie Heeter, during her time in Michigan State University, described this effect in The Communication in the Age of Virtual Reality as a principle based on a leap of faith and a transfer of self:
“Like Peter Pan thinking a happy thought, once you make the initial leap, reality becomes plastic and you can fly. Some people have an easier time thinking a happy thought than others…”
It comes as no surprise that in her research about one-fourth of all subjects studied were able to “engage belief” in a VR-based immersive experience. On the other side, another quarter of the sample was so strongly tied to the real world that they had difficulty immersing in the virtual one. With these two polar opposites, we naturally conclude that half of the subjects were “normal” in engaging belief, whereas they did believe they saw themselves in the virtual space and believed it to be true. However, it should be noted that this particular experience was conducted using a second-person perspective as opposed to first or third. Regardless, as designers we can concede that any experience we create should allow a majority of participants to suspend disbelief. Constantly test your world. Aim for presence in 75% of people or higher.
But suspension of disbelief is quite common, found in nearly every medium. Have you ever seen a movie in which midway through you question something? You mentally step away from the picture; perhaps a character performed an action that was out-of-place, or directly contradicted a presumption you had about them. Maybe a cut in editing crossed the 180-degree rule and you lose placement of the character’s positioning. It could even be a sound effect that didn’t sound natural or wasn’t mixed properly. In video games this is usually a bug or systematic error between player input and intentions. These moments are everywhere and we tend to notice it quickly. Suddenly instead of deeply engaging in a piece alongside the characters, we begin questioning a film’s world from our theater seat or the game’s world from our couch.
In film terms, this reaction is when you are “taken out of” an experience, and it is usually attributed to a design flaw. For games, it’s commonly a readability problem between what the game designer affords a player to do (their Horizon of Action) and what the player thinks they can do (Horizon of Intent). We will break these terms down in the next post, but please read Brian Upton’s TheAesthetic of Play or watch Read Me: Closing the Readability Gap in Immersive Games by Patrick Redding for when we get there.
As designers, we need to build experiences primarily for the 75% that can immerse themselves, and push to increase that number.
For film, the potential for design flaws lie primarily in Continuity. When a rule of the film is broken without reasonable cause, we no longer feel a part of that world. This can be as subtle as an anachronism or as noticeable as the self-repairing Porsche in Commando. There are plenty of other examples out there, and even modern films suffer an error or two if you’re looking for them. And that’s the thing, some people are always looking for them.
Even though Heeter’s findings were published nearly 26 years ago (and technology has vastly improved since), there’s still an important takeaway: Not everyone is easily immersed. Great immersive design is essential to capturing the engagement of an audience, but there will always be individuals unable to feel presence or immersion. This is okay. This is expected. Our job is to immerse the greatest amount of individuals out of the pool of potential participants, and build our worlds for the minimum 75% who can or could believe. Our endless goal is to give everyone the opportunity to fly.
Coming up next, I’ll go over the considerations when designing for realitative platforms, including methodologies and tactics surrounding the worldspace, narrative, and the connection between worldbuilders and their participants.