Every year, IBC gathers over 50,000 attendees to discuss the biggest challenges and triumphs of the broadcast technology industry. With huge industry announcements being made ahead of the show, such as the launch of the very competitively priced Apple TV+ service, along with the launch of Disney+, it was clear that the focus for this year’s show was certainly going to be centred on content. And while OTT was the topic du jour, my time at IBC showed me that leaps and bounds are being made in one particular area of content—immersive content.
Taking centre stage
While there was a lot of hype around this year’s keynote speaker, actor Andy Serkis, it was his ideas around immersive content that for me, stole the show. In particular, it was his vision for how immersive content will evolve storytelling that captured my imagination.
To give some background on how immersive content has developed to date—in the beginning, computer-generated imagery (CGI) evolved the art of storytelling by bringing characters that could previously only be interpreted through cartoons, pictures and models, to life. By paying close attention to texture, lighting and colour, CGI was able to make an animation look real, create familiarity, and thus create a deeper understanding between a character and the audience.
From CGI spawned motion capture technology, which captured the physicality of the actor’s movements, but with facial expressions animated from pre-recorded film, overlaid onto a character in post-production.
Today, motion capture technology has advanced into performance capture technology, capturing facial muscle movements, audio and physicality simultaneously. This evolution evokes a stronger reaction from the audience, increases their ability to empathise with characters, and thus takes viewers on an emotive journey.
Enter stage right: VR and AR
Having co-founded Imaginarium, a ‘laboratory-come-studio dedicated to exploring VR, AR and performance capture’, Andy discussed where the future of immersive content and the art of storytelling are heading.
He touched on how VR and AR will soon enable us to immortalise content and characters, reimagine figures from history and create theatrical performances that can be viewed by future audiences. Shakespeare, for example, will no longer be a dead physical figure from the past, but a virtual figure we can interact with in the present—this introduces a whole new facet to the art of storytelling. Instead of the directors and playwrights bringing their social context to stories and interpreting them accordingly, these stories will be told and performed in the way the writer originally intended.
The crescendo of Andy’s thoughts followed that soon, the audience will be able to step into the content and “wander around in the performance”—an evolution which might not be as far off as it sounds. Mobile and video R&D company InterDigital showcased V- PCC: The First MPEG Codec for Point Cloud Compression at IBC. It demonstrated real time point cloud decoding on a commercially available smartphone, which allowed users to step into the frame next to a 3D projection. Below you can see how I step into the content:
This notion of ‘stepping into content’ and viewers becoming immersed in content is likely to change the course of story-telling forever. It creates the opportunity for the audience to view performances exactly how they wish—not just viewing what the director films, but seeing what they want to see.
