IDEA has been hard at work creating documents and reference materials to put out and we now have an ITMF Road Map available for download with the ITMF Spec. Check it out.

A Memorable Year
Pete Ludé,  IDEA Chairperson

During the 2019 Holiday break, it would have been impossible to predict the monumental changes we all experienced during 2020.  On behalf of the IDEA team, we hope that you and your friends and family are navigating these challenging times and are ready for a road to normalcy (whatever that is) during 2021.  Despite the pandemic, there has been remarkable progress in the evolution of Immersive Media technology during the year, and IDEA is immensely proud to have been a part of it. Reflecting on the past twelve months, here are some of the highlights. First, we are proud to have nearly doubled our membership during the year.  We’re honored to welcome new members:

• Gridraster – a start-up providing the underlying compute and network stack to power immersive mixed-reality experiences on mobile platforms
• Looking Glass Factory– committed to building a headset-free, hologram-powered future.
• Pluto VR – whose is helping humanity transcend physical location, and
• Volumetric Camera Systems — a multi-camera solutions and production studio focused on providing next-generation immersive services for film, TV and beyond.

These remarkable innovators join existing members CableLabs, Charter Communications, Cox Communications, Light Field Lab, OTOY and Visby to round out a team of multi-disciplinary experts collaborating to invent the future immersive ecosystem.

Secondly, IDEA formed a new Working Group on immersive digital representation of live-captured images, acknowledging the potential differences between encoding photographically captured vs. computer generated holographic and light field content. In another milestone this year, IDEA member Charter Communications produced a remarkable demo at the October SCTE-ISBE Cable-Tec Expo – one of the worlds’ largest cable conferences.  Enabled by the contributions of fellow IDEA members CableLabs, Looking Glass Factory and Visby, the Charter demo was the world’s first live stream of a holographic imaging over a 10G cable network.  It involved one of the first on-location commercial light field production shoots using an array of over 100 cameras, with content processed and then transmitted over 10G cable network with minimal latency.  You can expect further real-life tests and demonstrations during 2021.

This summer, IDEA began the process of establishing a formal liaison relationship with MPEG to exchange information and nurture collaboration.  MPEG continues to make progress on the MPEG-I, Point Cloud Compression and other relevant standards which will compliment IDEA’s standards work.

Last month, IDEA participated in the annual conference of the Society of Motion Picture and Television Engineers (SMPTE), which was of course a virtual event.  Our team produced over two hours of content, divided into eight easily digestible segments, which are now available for you to view on the IDEA website, Click Here for Videos.   And perhaps more importantly, the IDEA Media Format Working Group released the updated version 1.0 of the Immersive Technology Media Format (ITMF) specifications.  Read more about what this means later in this news bulletin.  So overall – it’s been a busy and productive year.

Warm wishes for an immersive holiday break, and for a chance to spend some non-Zoom un-virtual time with your family.  The IDEA team is excited to roll full speed into the new year on the path to create an immersive media future, and we look forward to working with you!

The Future of Immersive Entertainment

By Eric Klassen, Executive Producer, CableLabs

The pandemic has accelerated the adoption of video communication technologies for interpersonal communications and entertainment at an unexpected rate. The idea of going to a screen in the home rather than doing something in-person has now been forced into day-to-day life for all generations and across all public sectors. This is a major shift in consumer behavior in an incredibly short amount of time.

When the threat of the pandemic is under control, most location-based experiences will revert to a normal routine: sports stadiums and airports will fill up, schools and office personnel will return, and concerts will have fans falling over each other in the front rows again. The difference, though, will be in the option of an alternative virtualized experience and the number of people that will opt for it. With ever-improving displays, compelling applications, and better network performance, many people may continue to work, learn, and play at home.

This consumer trend is not new. The great majority of football fans watch televised games even though going to the stadium is an available option: the high production value, amazing displays, rich sound, and avoidance of lines, crowds, and costly tickets makes staying home a compelling alternative. The pandemic has put movie theaters into a similar situation, and studios are being forced to re-visit how best to get new content to consumers without destroying the theater ecosystem in the process. For consumers to choose a private home experience over these location-based entertainment venues is a testament to how far media production, displays, and network technology has come over the last generation, and it’s only getting better.

Virtual Reality (VR) and Augmented Reality (AR) displays have come into market, and the experience of viewing an environment from a flexible viewpoint rather than one pre-determined camera position – known as “six degrees of freedom” or 6DoF – has a solid foothold mostly in the game industry. Although the applications outside of gaming are limited, developers are learning what works best, and gamers have become comfortable with spatial content and what can be done with it. A popular argument for 6DoF applications is that they’re a stepping-stone until better, more user-friendly spatial solutions find mass-consumer adoption.

Immersive displays using light field or holographic technology are no longer sci-fi and the ability to view volumetric spatial media, without a headset, is now available. Live action light field capture technology has been around for about a decade, but there’s never been a commercially available display technology that allows viewers to experience the changing parallax of the media without a headset. Up until now, the massive size of the media could barely run on local machines, and the idea of distributing it over the network was out of the question. Even if there was a display that could project the light field content, there was no way to get it to consumers unless petabyte-sized files could be downloaded and run locally. These limitations are on the verge of being overcome: early displays are out of the protype stage and showing promise, compression standards and network solutions are in development, and early use cases are paving a pathway to market.

However, some  challenges are surfacing for seemingly simple productions.

Imagine how a simple subject of a talking-head newscaster is seen on a standard display. When a newscaster speaks to the audience, they look directly into the camera, and the audience sees the newscaster looking directly at them. This single television view gives the same image to a viewer anywhere in the room.

With holographic displays, this is not the case. A viewer on the right side of a couch will see one side of the newscaster, and a viewer on the left will see the other side. This might seem like a trivial difference at first, but in terms of production considerations, this raises a series of questions. How will viewers react to sitting off-center from the newscaster’s gaze? If there are issues with this viewing experience, what can be done about it? How should multiple viewers in the room be handled? Should a multi-view function be designed that can be turned on and off?

If viewing positions can be identified in the viewing area, a new level of immersion will become possible, especially for gaming experiences. Imagine a smart character that looks directly at one player to say something, then turns to look at another player to say something else to that player. Or, imagine a magic scroll that hides the message from one player but another player in the room can see it. Game techniques like this have been touched on with headsets, but with light field displays, a new level of immersion is introduced that creatives are very excited about.

If consumers embrace holographic displays for improved telepresence, a critical development will come from better eye-to-eye contact. Currently, video telepresence is limited to a single camera view that’s slightly offset from the screen. Today, complex image processing software is required to correct the offset eye-gaze, but light field capture would provide the needed data to engineer one-to-one, one-to-many, and many-to-many social gatherings with more natural eye contact.

As display, capture, and network technologies improve, the future is ripe for a holographic media ecosystem. What does a holographic experience in the home and office look like in ten years and beyond? Is there a holodeck in sight? The Immersive Digital Experiences Alliance (IDEA) was founded with leaders in this ecosystem to imagine what this future will be and to work towards realizing it.

Did you miss IDEA at SMPTE 2020? No worries, IDEA’s SMPTE Videos are now available here on our site!