Why HDR matters more in VR and AR
High Dynamic Range (HDR) is a term that is used in relation to describing the increased area of variation in both color and brightness made possible by displays that support it.
HDR comes in a confusing handful of standards, HDR10, Dolby Vision, Hybrid Log Gamma, Advanced HDR and most recently HDR10+. What matters is that all of theses standards are mapping colors across an increased ‘color space’.
In the image above the smaller ‘HDTV’ triangle represents the colors that can be displayed using the ‘Rec. 709’ color area standard that was developed in the mid-to-late 1990s, for Standard Dynamic Range displays. The larger ‘UHDTV’ triangle represents the
‘Recommendation ITU-R BT.2020.’ color area (Rec. 2020) that HDR televisions aspire to cover.
But wait! The image you are looking at is not an HDR image, and the display you are reading this on may not be an HDR display, your computer/phone is probably not rendering to a HDR framebuffer (10 bits or 12 bits per RGB channel instead of 8), and your browser is probably not capable of displaying HDR content. So the colors you are seeing above are in reality all being mapped within the ‘Rec. 709’ standard color area, the smaller triangle.
If you could see the image in HDR it would look much more vivid. But since you cannot, lets at least examine quantization in relation to the ‘8 bits per channel RGB frame buffer’ that displays use. 8 bits yields 2^8 = 256 possible values per channel, which makes ‘256 reds * 256 greens *
256 blues’ possible. This results in a total of 16,777,216 possible colors. That sounds like a lot!
And yet actually it is only going to contain 256 possible grey-scale values (0, 0, 0 through to
255, 255, 255). If you look carefully at the image above on a large display, you’ll see vertical
banding – this is where the step up from one grey to another is too big, and rather than seeing a smooth gradient we see one band changing to another, in the same way that a flight of steps is not as smooth as a ramp.
Since Rec. 2020 covers a larger color space it also requires either a 10 or 12 bits (yielding
1024 or 4096 possible values) per channel granularity, otherwise banding would increase since the RGB boundary colors are now physically further away.
To visualize why this additional granularity of values is needed, here is the color gamut image from above mapped using just 256 values, instead of 16.8 million. Notice how severe the banding has become. Dramatic ‘steps’ are being taken in terms of the change between colors.
Brighter displays allow for a more realistic representation of the world because in the real world the sun measures around 1,600,000,000 nits, and the night sky around 0.001 nits, and average
modern TVs max out at around 500 to 1,000 nits (Source CNET article). Samsung’s S10 phone supports 1,200 nits peak luminance.
But, as the displays get brighter the same issue of color quantization requiring a deeper bit depth applies to brightness also.
Why does this matter more in VR and AR?
In a movie theatre they turn the lights off because the screen may only be 50 nits. So they are trying to remove all other ambient light, so that your eyes will adjust and make the most of the low level of luminance on the screen. This can be disappointing, in comparison to an HDTV or a phone which is much brighter. The projected image can look washed out, with black levels
that are not dark enough, and bright colors that are still are not as vivid as the colors on your phone.
In both virtual and augmented reality, the fidelity of the simulated experience, whether real or fantastical has a greater impact, because it envelopes us visually. The screen we are looking at is so close to our eyes that imperfections are made more obvious – the flaws are magnified. Conversely this means that improvements to color depth and luminance range yield greater results than on a display further away from your eyes, surrounded by ambient light.
Resolution determines how many pixels the display has.
And more pixels are better to a certain point, just like the volume on an amplifier makes it better at playing room scale music than a phone can be.
But volume in itself cannot improve the quality of the recording! A low bitrate MP3, or a cassette recording will not produce high quality audio, just because they are played loudly.
Fundamentally although 8 bits per channel RGB Rec. 709 images are ubiquitous and what we are used to, they are as inappropriate to VR and AR as cassettes are to music.
To understand why HDR is important, particularly in VR and AR, remember that computers used to run in 256 color mode.
In the past this was normal, and acceptable.
Very few people do that anymore, because it is clumsy and limiting.
I believe that the same conclusion will be reached regarding Rec. 709 SDR images.