Immersive Technologies are here to make their mark in almost every industry! But then, let’s admit it, our bodies were not designed for Virtual Reality.

By applying artificial stimulation to our senses, we are disrupting the operation of biological mechanisms that have taken hundreds of millions of years to evolve in a natural environment.

We are also providing input to the brain that is not exactly consistent with all of our other life experiences.

And this is why, there are also many timeswhen our bodies react with fatigue or headaches, partly because the brain is working harder than usual to interpret the stimuli.

The Real Dangers

I recall the very first time I experienced this when I put on the VR headset and entered ascene with no floor! It seemed as if I was floating in the middle of nowhere . What is interesting is that, I was so hesitant in moving from that one point.It was as if my brain was trying to make sense of all of it and while it did that, it put all the known responses to action anyway, standing still being one of those.

The other kind of confusing stimuli is through roller-coaster rides in VR. One can get an onset of sickness, which typically involves symptoms of dizziness and nausea. Now I am not a very big enthusiast of any fast moving, disorienting ride. When I  tried the roller coaster in virtual reality, it was surreally real. My heart beat increased and I was terrified. All the visual signals that the brain received were making it believe that there was danger that lay ahead, and the brain activated all the saved responses.

However, just as I closed my eyes, I felt safe. There was no other stimuli of roaring sounds, fast movement, unbalanced posture, or more to the fact, anything.

Perceptual Psychology

Perceptual psychology is the science of understanding how the brain converts sensory stimulation into perceived phenomena.

It is an extremely important field for virtual reality designers, application/game developers and similar professionals to understand what works in the virtual world.

Tricking the brain is not easy and when in a virtual reality world, questions like this become crucial when designing such experiences-

• Why does one feel nauseated?
• How far away does that object appear to be?
• How much video resolution is needed to avoid seeing pixels/ smudges?
• How many fps (frames per second) are enough to perceive motion as continuous?
• Is the user’s head appearing at the proper height in the virtual world?
• Where is that virtual sound coming from?
• Does my feet look like it is on the ground?

To answer these questions and more, we must understandbasic physiology of the human body, including sense organs and neural pathways, AND the key insights of experimental perceptual psychology.

The perceptual side of VR often attracts far too little attention among developers.

In the real world, perceptual processes are mostly invisible to us. Thanks to years of evolution and the way our brain is conditioned to fill in the gaps for us.

Let’s see how.

Optical illusions

One of the most popular ways to appreciate the complexity of our perceptual processing is to view optical illusions. These yield surprising results and are entirely unobtrusive. Each one is designed to reveal some shortcoming of our visual system by providing a stimulus that is not quite consistent with ordinary stimuli in our everyday lives.

These should motivate you to appreciate the amount of work that our sense organs and neural structures are doing to fill in missing details and make interpretations based on the context of our life experiences and existing biological structures.

What do you see here?

It is random smudges until it is pointed out that this picture contains a Dalmatian dog.

Suddenly your brain finds patterns and fills in the missing edges to make sense of the whole composition.

Some people can even identify road, fence and garden.

Tihs is aonhtrscuhexmalpe.

As far as the fsirt and lsatlteter is in palce, our barinraeds it as a flul word.

Fun fact I– There is always an area of the thickness of a pencil that is dark in our field of view. This is the area on retina from here the optic nerve bundle leaves from. Why we don’t see this ever, is because our brain fills this up for us, basis what is adjacent to this spot in the scene.

Fun fact II – During the millions of years of evolution, our brain has evolved to get us ready for the next. What we see around us is not what is happening at this moment but one-fifth of a. second later. We are fed a future feed through eyes to be ready and proactively take certain actions, which could be lifesaving.

The Actual Senses

Perception and illusions are not limited to our eyes. Let’s understand human physiology a bit more.

In our bodies, a stimulus is converted into neural impulses.

• In each eye, over 100 million photoreceptors target electromagnetic energy precisely in the frequency range of visible light.
• The auditory, touch, and balance senses involve motion, vibration, or gravitational force; these are sensed by mechanoreceptors.
• The sense of touch additionally involves thermoreceptors to detect change in temperature.
• Finally, our sense of taste and smell is grouped into one category, called the chemical senses, that relies on chemoreceptors.

When we see the world around us – we feel it with all of our senses. There are hundreds of billions of signals that our eyes send to the brain every second. But that’s only a part of the story. When you ‘see’ something only 10% of  the information comes from optic nerve.

There are other critical parts of the brain that have to deconstruct the signals – recognize faces, interpret movements, identify feelings, identify danger and act accordingly.

Which implies that the biggest part of what we see is not through visual signals, its making sense of them in the brain.

This is where the brain gets tricked.

Let’s talk about relative positioning that plays an extremely crucial role in virtual reality.

Proprioception – The Relative Positioning

In addition to information from senses and memory, we also use proprioception, which is the ability to sense the relative positions of parts of our bodies and the amount of muscular effort being involved in moving them.

To understand this, lets’ do a small activity – close your eyes and move your arms around in an open area. You should have an idea of where your arms are located, although you might not be able to precisely reach out and touch your fingertips together without using your eyes. This information is so important to our brains that the motor cortex, which controls body motion, sends signals called efference copies to other parts of the brain to communicate what motions have been executed. Proprioception is effectively another kind of sense.

Fun Fact III – One interesting implication of proprioception is that you cannot tickle yourself because you know where your fingers are moving; however, if someone else tickles you, then you do not have access to their efference copies. The lack of this information is crucial to the tickling sensation.

Fusion of Senses

Signals from multiple senses and proprioception are being processed and combined with our experiences by our neural structures throughout our lives. In ordinary life, without Virtual Reality, our brains interpret these combinations of inputs in coherent, consistent, and familiar ways. Any attempt to interfere with these operations is likely to cause a mismatch among the data from our senses.

The brain may react in a variety of ways to this conflicting stimulus. And this is when all the fatigue, headache or even dizziness / nausea start. In certain cases, the brain might react by making us so consciously aware of the conflict that we immediately understand that the experience is artificial.

So how to get enjoyable immersive experience?

To give that amazing immersive experience, we still need to trick the brain – to have it by-pass all the responses to known & unknown stimuli, it has stored in its records.

And to make that effective and comfortable VR experience, designers and developers need to keep all the aspects of human physiology into consideration while designing virtual worlds.

One of the reasons ‘Beat Saber’ has been trending as the number#1 game in virtual reality is because it has most of the questions sorted. This is a VR rhythm game where you slash the beats of adrenaline-pumping music as thee cubes fly towards you, surrounded by a futuristic world.

• The floor keeps you ‘grounded’ – Bingo! You are on the same planet.
• The cubes come towards you in a linear unidirectional way and the rest of the scene remains as it is and therefore, there is no extreme sensory overload.
• Music comes through places near your ears providing spatial balance.
• Arm movements and head tracking are in line with real life scenarios.
• Lastly, the overall experience is comfortable.

This is obviously in the standard, straightforward mode – Easy or Normal. And when someone feels the need to be pushed or to test themselves even more, they can move on to much more advanced levels such as a 90 and 360 degree orientation where the cubes approach from across the angle.

Another game that I recently tried for the first time, involved me zooming in towards the whole scene at a maddening speed. Now that is something I can’t control and got really uncomfortable with, very soon.

I couldn’t even play it for a minute and had to close my eyes!

 

Another example is through VR social application called Altspace VR.

This is a virtual platform for people to meet others, play games together, attend live shows, have open mic sessions, have party and basically socialize with people from all over the world.

And like Beat Saber, it has got most of the questions sorted-

• Your feet is more or less always on the ground -unless, let’s say, you opt to go stand on top of a bench
• Your position, the speed of walking or jumping to a point is controlled by you. You get to decide where to stand, your personal space and proximity to others.
• You can get to hear music near stage or places where its not ambiguous. Like there is no music on basketball court.
• Arm movements and head tracking are in line with real life scenarios.
• Lastly, the overall experience is quite interesting.

Yet, they can improve certain aspects of motion so that it looks more like walking than gliding, stepping up and down rather than floating when moving to a different height, fist bumping/ handshaking more real etc.

That’s the thing.

The user can now control how much ‘tricking’ their brain needs to feel the thrill.

And till the time our brain thinks that it is in control, we will be comfortable in any immersive experience.

 

Applications–

Tiltbrush

Beat Saber

YouTube VR

Anne Frank VR

AltSpace VR

 

References–

Essentials of Sensation and Perception (Foundations of Psychology) – George Mather

The Body – Bill Bryson

Virtual Reality – Steven M LaValle

 

Researched, compiled and written by –

 

Bhumika Sharma, PMP, CSM

Senior Manager,

Publicis Sapient, Bengaluru

Member of VRARA

https://www.linkedin.com/in/bhumikasharma/

About Blog Writer :

---

Bhumika Sharma

Senior Manager

Publicis Sapient, Bengaluru

Member of VRARA