Voices of VR

• #1558: Series Director of “Adventure” Series on Apple Immersive Video and Grammar of 180 Video

  Charlotte Mikkelborg is the series director of the Adventure series on Apple TV, which is shot on Apple Immersive Video and co-produced by Atlantic Studios (formerly Atlantic Productions) and Apple. I think it has the strongest storytelling released by Apple so far, and it also has some of the most innovative and cutting edge shots that really show the power of the 180-degree format of Apple Immersive Video. In this conversation, we cover Mikkelborg's 10-year journey into working with immersive formats starting with 360-degree video, multi-sensory VR, and most recently 180-degree video. We also talk about the emerging grammar of 180-degree filmmaking, and the introduction of close ups, which I feel like work sometimes and other times feels almost too close or can feel a bit uncanny or jarring when changing scales or results in warping or other distortions. We also talk about some of the behind-the-scenes insights about the Adventure series. One concerning thing that I noticed after watching all of the Apple Immersive Videos released so far is that many of these videos do not contain any credits for who worked on these projects. I did an audit of all 22 of the published Apple Immersive Videos, and I found that 14 of the 22 don't have any credits at all,. Most of these are by Apple Sentity LLC productions, which is presumably Apple's in-house production team. I found that 5 out of the 22 have partial credits, including the four episodes of the Adventure series as well as Man vs. Beast, which features DGA director Ryan Booth. And there are 3 out of the 22 Apple Immersive Videos that have full and complete credits including Submerged that features DGA director Edward Berger, and the Prehistoric Planet Immersive series, which is a co-production with Fairview Portals. See the table below for more information: It is deeply concerning to me that only 3 out of the 22 Apple Immersive Videos seemingly have full and complete credits. Apple did not provide any comment about why this is or why some of these pieces have no credits, why some have partial credits, and why some have full credits. It is nice to see at least some credits on the Adventure series, which is a co-production between Apple and Atlantic Studios (formerly Atlantic Productions), but they're only listing the director, executive producer, and key talent while all of the other key creative talent are not being credited. Atlantic Productions did not provide any official comment about why there are only partial credits, while a few other productions have full credits and many other productions have no credits. You can see find a partial a list of some of the Adventure series creative talent on IMDB, and episode 3 protagonist Ant Williams posted a partial list of credits on his Instagram announcement post. I was able to get some additional context from Mikkelborg about the credits situation, and it does sounds like the co-productions likely have a bit more leeway in dictating what types of credits are shown. While the Prehistoric Planet Immersive series co-produced by Fairview Portals series has full and complete credits, and the Adventure series by Atlantic Studios only has partial credits. The two Apple Sentity LLC productions that do have credits also happen to be directed by directors who are a part of the Director's Guild of America (DGA). The DGA has specific credits requirements, "For feature films, the Director’s screen credit must be accorded on a separate card, which shall be the last title card appearing prior to principal photography. This credit shall be no less than 50% of the size of the displayed title of the motion picture, or of the largest size in which credit is accorded to any other person, whichever is greater." The rules may be different for immersive productions or emerging media projects, but the director credits for Submerged and Man vs Beast seem to both meet DGA's credits requirements for film.

  --------  

  59:40
• #1557: Apple Immersive Video Behind-the-Scenes & Overcoming Fears with “Adventure” Series Athlete

  Ant Williams is a freediving athlete featured in the third episode of The Adventure Series titled "Ice Dive" that's co-produced by Apple and Atlantic Studios (formerly Atlantic Productions). In the episode, Williams attempts to swim a world record distance 182 meters under ice, and I wanted to get some additional behind-the-scenes context on his experience as well as what it was like to have the most intense month of his life condensed down into a 15-minute Apple Immersive Video. Williams is a sports psychologist who wanted to put his theories into practice by taking what he calls "positive, calculated risk-taking challenges" that allow him to deal with overwhelming anxiety, and overcome his fears, uncertainty, and self-doubt. I also wanted to get some additional context on the production of the episode as Apple has otherwise been pretty tight-lipped about the series, launched with which launched with "Highlining" on the same day as the Apple Vision Pro launch on February 2, 2024. Apple Immersive Video is a different format than spatial video. Apple says "spatial videos are captured in 1080p at 30 frames per second in standard dynamic range," and these are what can be captured by either an iPhone or Apple Vision Pro, and they are displayed in a windowed frame where you see the stereoscopic effects. Apple describes Apple Immersive Video as "a remarkable storytelling format that leverages 3D video recorded in 8K with a 180-degree field of view and Spatial Audio to transport viewers to the center of the action." Apple Immersive Video is much closer to what we've seen from the XR industry and VR 180 filmmakers from the past decade, and Apple's technology is likely derived from their 2020 acquisition of NextVR. NextVR focused on live stereoscopic broadcasts of sports events on VR headsets starting with the Samsung Gear VR and Oculus Rift in 2014. A lot of the technical specifications of the Apple Immersive Video format have not been officially confirmed by Apple, but there are a couple of breadcrumbs that give us some more details. Thanks to iFixIt's breakdown of the Apple Vision Pro on February 7, 2024, then we know the microOLED display size is reported as "the lit area totals 3660 px by 3200 px." 360 Labs' Mike Rowell wrote a post on March 19, 2024 saying, "Apple Vision Pro’s screens are a whopping 3660 x 3200 pixels per eye. Although they haven’t made any official claims as to the FOV of the headset, 3rd party developers claim that it looks to be around 100° horizontal. With each screen having 3,660 horizontal pixels, this would mean that a 180° immersive experience would need about 6,000 x 6,000 pixels per eye to saturate the display. Apple’s own immersive experiences have been reported at being 4320x4320 per eye at 90fps and in HDR10." The reporting of Apple's immersive experiences was detailed by Mike Swanson, who announced a spatial video tool on March 7, 2024 that leverages the Apple's AVFoundation to properly encode video into the "multiview extensions of the HEVC codec, known as MV-HEVC" format. Swanson says in his post, "I receive multiple messages and files every day from people who are trying to find the limits of what the Apple Vision Pro is capable of playing. You can start with the 4320×4320 per-eye 90fps content that Apple is producing and go up from there. I’ve personally played up to “12K” (11520×5760) per eye 360-degree stereo video at 30fps." Another clue can be found in the Blackmagic URSA Cine Immersive camera that was announced on June 10, 2024, which says, "The sensor delivers 8160 x 7200 resolution per eye with pixel level synchronization and an incredible 16 stops of dynamic range, so cinematographers can shoot 90fps stereoscopic 3D immersive cinema content to a single file." Incidentally, Currents director Jake Oleson told me that he used Swanson's tool to create his immersive film after shooting it in 8k on the Canon EOS R5 Camera Body with Canon's Dual Fisheye lens.

  --------  

  53:32
• #1556: “Currents” Boldly Defines a New Era of DIY 180-Degree Immersive Filmmaking with a Kit Under $5k

  CURRENTS is a stunning debut from Jake Oleson as it pushes the grammar of 180 filmmaking to new heights. Oleson comes from the world of advertising and music videos, and he went through a rigorous pre-production process where he had actually already edited the entire piece before shooting any production footage by using still stereoscopic test photos and videos. The insights from this pre-production animatic are evident, as there are some of the most stylized movements I’ve seen, and he managed to not trigger any of the common motion sickness triggers. There are some really powerful cuts that seamlessly juxtapose different environmental contexts for dramatic effect. CURRENTS tells the story of a young woman who leaves her Vietnam rural countryside home to the city seeking more economic opportunity. Aside from a short exchange with her mother at the beginning, the rest of the entire story is told through spatial storytelling, driving electronic music that was also composed by Oleson, and a shader-filled point-cloud scene with some animated motion capture. It's really quite a beautiful piece that gave me a glimpse of the next phase of cinematic immersive filmmaking. It's also worth mentioning CURRENTS in the context of other musical experiences since Oleson is both a musician who has a background in music video production. The music was specifically composed to serve the narrative in this instance, but it will be interesting to see more and more immersive music videos start to crop up now that the Apple Immersive video format is starting to get licensed out to folks like Vimeo, who commissioned this piece. Apple's push for 180-degree immersive video has brought up some broader discussions about the merits and downsides to 360-degree films. From a creator's point of view, 180-degree filmmaking is a lot easier to do logistically as it is closer to existing production pipelines, which is brilliantly demonstrated in CURRENTS. It's also worth noting that CURRENTS has a point-cloud scene that shows up in the middle of their cinematic 180-degree immersive video that represents a key turning point in the story and journey of the main protagonist. There's also some really poetic shaders that are translating the static scenes into even more of a dynamic visual representation of the business and chaos of the city. It has quite a narrative impact when juxtaposed against the photorealistic footage and custom-written and driving musical score. Even though the Apple Vision Pro has been out for over a year now, SXSW is still an opportunity for folks to check out the Apple Vision Pro for the first time. It was probably easiest to check out on CURRENTS who had six different headsets and was using the Sandwich Vision Theater app in Kiosk mode to seamlessly jump into the experience bypassing the time-consuming eye tracking calibration step. I was also particularly interested in Olson's production pipeline, the camera gear and kit costing less than $5k (Canon EOS R5 Camera Body, RF5.2mm F2.8 L Dual Fisheye, & DJI Ronin-SC Gimbal Stabilizer), Topaz Video AI Enhancer upscaling tools to go from 8k to 16k, and all of the due diligence that he did to understand and mitigate motion sickness triggers. The end result feels like a music video that tells a timeless story of the tension between rural and urban homes and what is lost in the pursuit of career opportunities. Below are links to of all of my SXSW 2025 coverage by the Voices of VR podcast. And be sure to check out this Linked article: Recap of All SXSW XR Experiences Including Voices of VR Podcast Interviews with Every Artist. SXSW XR Experience Competition [Overview of all of the SXSW 2025 XR Experience Program] - #1529: SXSW 2025 XR Experience Selection Sneak Peak with Programmer Blake Kammerdiener 1906 Atlanta Race Massacre - #1548: 1906 Atlanta Race Massacre Uses Phone-Based AR to Volumetrically Tell a Forgotten History

  --------  

  57:28
• #1555: Award-Winning Tabletop Animation “Oto’s Planet” Uses Unique Interactive Mechanic to Chose Perspective

  OTO'S PLANET was being exhibited on the Quest at SXSW, but I had a chance to check out the Apple Pro Version ahead of the SXSW festival and I felt like I much preferred it. OTO'S PLANET picked up the second place prize at Venice Immersive, but I missed being able to interview the director Gwenael Francois there. I did manage to catch up with Francois at SXSW to talk more about the development of this beautiful interactive narrative that uses table top-scale animation, and requires the user to rotate a tiny planet around in order to follow the story of Oto, his pet, and the story of a colonizing intruder who arrives. I found that the interactions were much smoother and immersive on the Apple Vision Pro than the Quest, and the higher resolution also popped a lot more. The Apple Vision Pro version defaults to mixed reality while there is an opportunity to see it in fully immersive VR on the Quest. I normally would prefer the Quest version, but having the better quality mixed reality cameras and overall much higher resolution in the Apple Vision Pro. Apple also featured Oto's Planet, and DPT CEO Nicholas Roy shares in this interview that they've actually received more purchases on Apple devices than Meta's ecosystem. Meta’s storefront has been flooded by Meta's own first-party Horizon world as well as App Lab apps, and so it’s very telling and reflective of the current state of distribution to have more sales for the Apple Vision Pro than Meta Quest. It is promising to see Apple get more serious at immersive storytelling as there was a session titled "Create Interactive Stories for Apple Vision Pro" on Tuesday morning featuring three representatives from Apple. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

  --------  

  41:48
• #1554: “Cosmos in Focus” Contextualizes James Webb Telescope Images in Educational Immersive Planetarium Experience

  Another Apple Vision Pro piece at SXSW 2025 was COSMOS IN FOCUS by Atlantic Studios (recently rebranded from Atlantic Productions). This was a short and sweet educational experience that allowed you to explore the imagery of the James Webb telescope, but it's both spatially contextualized in the night sky planetarium style. It is also fused together with other deep space satellite images at different scales of resolution creating this really cool zoom effect that's like a mash-up of the film POWERS OF TEN with Google Maps tiles, but in the context of space imagery. It's also an experience that demonstrates the power of collaborating with scientific researchers and subject matter experts as they're able to translate contextless 2D jpgs into an incredibly powerful immersive experience that not only preserves the original context, but leverages the immersive medium to provide many more learning opportunities. Many immersive stories on the festival circuit will shy away from more explicit didactic content or learning experiences, but this piece leans into it by reducing the choices a user can make streamlining the user journey. I also really enjoyed my conversation with Aditi Rajagopal, who led this project after transitioning from R&D and AR storytelling at Meta to immersive storytelling at Atlantic Studios. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

  --------  

  44:37

More Technology podcasts

Trending Technology podcasts

About Voices of VR