Crafting a compelling documentary is a difficult endeavor for any artist under normal circumstances. When you factor in the constraints of a global pandemic of 2020, it seemingly becomes an unachievable goal.
For Emmy nominee Gary Yost, that was far from the case as he crafted his fascinating documentary, Inside COVID19. With its immersive 3D animation — captured through an Oculus VR headset — Inside COVID19 chronicles the erratic journey of the COVID19 pandemic.
Yost’s previous documentary, The Invisible Peak , brought the filmmaker awards for Best Documentary, Best of Show, and Outstanding Environmental Vision on its festival run. This year, Yost hopes to claim Emmy gold.
Outside of filmmaking, Yost’s talent for software development is well-known in the business. He created one of the most widely-used 3D production systems in the world, Autodesk 3ds Max, and recently launched the non-profit WisdomVR Project. WisdomVR aims to create a living library of immersive cinematic experiences and an open source, global educational platform.
Awards Focus spoke to Yost about his journey from photographer to award winning director, the intricacies of being a software creator, and where he sees VR heading as billions are invested into the new technology.
Awards Focus: You seem to have so much going on in so many fields. Can you talk about the path that led you to pursue software design as a filmmaker and photographer?
Gary Yost: My teens and early 20s were filled with photography and filmmaking experiments – animation was always a big part of that, mostly time lapse and stop motion. The film Tron was released in 1982 and seeing its mind-bending animation created with computers blew me away.
That same month the Timex Sinclair 1000 microcomputer started shipping for $99 so I bought one and started teaching myself how to code. Robert Abel was a big force behind the 3D CGI in Tron and then also in 1984’s groundbreaking “Sexy Robot” Canned Food TV spot.
By that time I was hooked but couldn’t possibly afford the hundreds of thousands of dollars to purchase Silicon Graphics hardware or the new 3D software from Wavefront Technologies (founded by a crew from Robert Abel’s company). Thus began my 15-year adventure creating a brand new generation of PC-based 3D content creation tools, culminating with selling our IP and patents to Autodesk at the end of the century. I was done with the never-ending software release cycle (which we called the “deferred life plan”), had put in my time creating a new genre of affordable 3D VFX tools and was itching to get back to my first love… photography and filmmaking.
AF: How did your success in software design influence you to create WisdomVR?
Yost: The process of inventing Autodesk 3ds Max was rooted in how our incredibly talented small team would solve complex software development challenges on a daily basis. We were constantly breaking new ground – never knowing what novel issues would arise but always feeling confident that my team could address them. I’m not sure if you’re aware of this but my team of six software engineers was competing with much larger teams (such as the order-of-magnitude larger team at Alias) and we were outselling their software 10-to-1. We were all working remotely, operating at peak efficiency and our confidence and hardcore work ethic powered us to solve very difficult issues quickly, many of which had never been solved before.
With WisdomVR we’re doing something very similar. Again I’m leading a handful of super-talented creatives tackling a new medium that hasn’t been mastered by anyone. VR360 documentary experiences are completely unlike traditional films… they put the viewer into the scene, literally embodying them as a voyeuristic being inside the story. There’s no concept of a close-up, medium or wide shot… it’s all just there in front of you and the story has to come from the art of weaving the location and the subject together. That would be terrifying to most traditional flat filmmakers but for me, since I’ve built so much confidence in breaking new ground from my 3ds Max experience, it’s incredibly exciting.
The bottom line is that building a team of brilliant engineers to create 3ds Max is not much different than building our team of directors, animators, spatial sound and vfx artists to create WisdomVR.
AF: What inspired you to make Inside COVID19 a VR film rather than a regular film?
Yost: Our mission is to capture personal stories using this amazing medium of presence with the potential to connect in the deepest way possible. The idea behind all of our work is that after you’ve seen it and take the headset off, you feel more connected to the world than you were before. That’s the core reason why we’re not making regular films… connection.
The story of the pandemic is in many ways a story of isolation, confusion and our search for meaning within the past 18 months of chaos. With traditional 2D filmmaking, viewers watch a story unfold on a screen (that these days is usually very small), giving them lots of opportunities to be distracted by other things happening in their lives.
With stereoscopic immersive VR360 filmmaking, viewers are completely surrounded by the subject’s life with no opportunity to look away. This brings an unrivaled intensity to the experience and with Inside COVID19, witnessing Dr. Child’s life is so powerful that some people have told us that it has almost too much impact for them… it’s TOO intense. No other medium can do that and we use that intensity to cut through the confusion.
When you experience this story about the pandemic in a headset, completely surrounded by hyper-real stereoscopic 360-degree imagery, you become a true voyeur… a being embodied inside the story. The impact of this experience is impossible to convey with words, and thankfully there are many Oculus Quests, Rifts, and Vives now out there and many people either have a headset or have a friend they can borrow from.
VR films transport you directly into the subject’s world and we capitalized on how open Dr. Child was, giving us access to his work, home, family and even his private CT scan data. We used those scans to create 3D animations, flying deep inside his chest to see the massive damage COVID-19 caused in his lungs… can you get more intimate than that?
As you sit in his living room with his wife and two children as they directly share his near-death experience, it’s clear that VR provides the highest level of emotional impact.
Many people might not know someone who’s been so deeply affected by COVID-19 but after watching our piece you’ll come away from it with the visceral feeling that you do. In what other medium can you become embodied inside one of your epithelial cells and watch a virus enter and begin replicating right in front of you?
AF: What impact do you hope Inside COVID19 leaves on history?
Yost: Inside COVID19 is a detailed record of a physician’s experience both treating and suffering through what will be remembered as a pivotal moment in modern human history. By presenting a step-by-step recording of how the pandemic evolved in 2020, we’re helping future generations remember exactly how little we knew when all of this began and how rapidly the global community came together.
At the start of the pandemic we knew that the original mission of WisdomVR’s project to preserve elders’ memories was more important than ever, but like so many we were at a loss for how to safely continue or what to do next. My directing partner Adam Loften and I were both very concerned about the mysterious news from China and by the time the virus arrived in the U.S. we were compulsively trying to learn as much as possible to stay safe. Beyond the fear, it was also a fascinating moment and once we met Dr. Child, we knew that he could be our guide through this tremendously challenging time.
We believe the best path into any topic is through a personal story. The story of this pandemic is huge, but the medium of immersive video is intimate. When we have a personal connection to someone who’s suffered, it orients us into the pandemic in a way that no other experience can. That moves people to take steps to keep themselves and their families safer because it’s not an abstraction anymore. Our main hope is that this film will help reinforce public health messaging and drive vaccine acceptance.
The global tragedy has filled us with urgency that this is a transformative moment. That urgency itself inspired us and Inside COVID19 is our contribution to this transformation. It is also the only immersive record of the pandemic yet made, and the only experience in which you can both shrink down to the molecular level to watch the SARS-CoV-2 virus replicate in one of your cells and then zoom out into space and look back on our beautiful planet.
AF: As the founder of WisdomVR, how do you imagine the project emerging in society? What will this look like?
Yost: VR headsets are finally affordable, mobile and becoming ubiquitous – two huge players are pushing hard on it. Facebook’s Reality Labs/Oculus division now has 10,000 people with an $18 billion a year budget and Apple is also spending billions per year on their own upcoming VR ecosystem. While there’s a ton of focus on games and 6DoF synthetic experiences, in the immersive documentary film space there are only a handful of people focusing on making high-quality VR360 content.
WisdomVR believes in this new art form because it allows unmediated experiences of people and places, preserving them in situ for generations to come. Understanding the highest potential of this storytelling medium is something I’m devoting the rest of my life to.
As millions of new headsets are entering people’s lives around the world, a mass environmental and cultural extinction is simultaneously taking place. VR360 is an opportunity to preserve and share precious people, places and cultures, especially those that are most vulnerable. WisdomVR is dedicated to two things: 1) refining the tools to create the highest-quality impactful VR360 documentaries and 2) sharing everything we learn along the way so that others can preserve their own precious heritage and cultures. There’s a huge gap in the VR space for these types of humanities experiences and we want to change that. The audience of viewers has a hunger for meaning and even if early adopters aren’t conscious of it now, this immersive medium will drive a new wave of humanities and social impact.
Our long-term goal is to create a living library of immersive cinematic documentary experiences where people can sit with amazing people like Dr. Child and have a 1-to-1 communication of wisdom. In addition to creating discussion guides for each experience, we are also developing an open source VR360 heritage documentation framework for the public good.
Precious stories are being lost and we can help to make sure that they’re not forgotten. We hope to have many collaborators – our nonprofit’s mission is to share everything we know and we’re openly looking for partners to make all of this possible.
We are all born storytellers, dependent upon our cultures to carry our evolving wisdom to future generations. New tools for the preservation of the wisdom of the world are emerging in a library that anyone can experience and add to, as well as a curriculum where people can use the benefit of our experience to preserve their own heritage. Inside COVID19 is one part of the bigger library that we’ve been creating since 2018.
AF: What was the most challenging part of displaying the replication of the SARS-CoV-2 virus through immersive 3D animation?
Yost: A virus is invisible to the eye, and to most it’s just an abstract idea, but 3D animation is perfect for illustrating exactly this sort of thing: processes that are either too small or too large to show with video.
We had to learn how to portray the molecular processes that describe how the virus gets into our cells and replicates. The idea to do this began from seeing the now-famous 3D SARS-CoV-2 model that Alissa Eckert and Dan Wiggins at the CDC built in late March and reading about how they created it with 3ds Max, the software I’d invented. I figured if they could do it, we could do it and it gave me the inspiration to use Max to show people something they’ve never seen before… a fully animated VR visualization of the molecular viral world, from entering our nasopharynx to becoming invaginated in one of our epithelial cells. So challenge #1 was to understand the science, and luckily Alissa connected me with David Goodsell’s work at Scripps Research Institute and we used his amazing illustrations as a roadmap.
Our animator Andy Murdock has over 10 years of experience creating scientific animations for National Geographic, and our storyboard was ambitious! In two months, we had to produce seven minutes of 6720×6720 3D/360° animation designed to accurately represent the complexity of the SARS-CoV-2 virion’s lifecycle. So challenge #2 was how to render three billion pixels of information for each second of seven minutes total… 180 billion pixels per minute or 1.2 terapixels of pi. Just as a comparison, 7 minutes of 3D CGI HD monoscopic video would only require 1/24th of that data and typically even 7 minutes of 3D would be challenging for most indie studies to render in such a short time. But I have friends with benefits from my CGI days and we were granted a tremendous amount of free rendering time on the massive Chaos Cloud render farm. Plus there were brand-new shader techniques for creating the complexity we were looking for without using insane amounts of geometry… an example is the phospholipid bilayer membrane around the epithelial cell – incredibly accurate and yet produced entirely with a shader!
With Emmy voting starting just around the corner, what would awards recognition for Inside COVID19 mean to you?
The true stories that we tell each other have the capacity to make us safe and strong… these stories have meaning and help people become resilient in the here and now. Our WisdomVR stories, both cultural and scientific, contribute to a strong and healthy society.
Awards recognition for Inside COVID19 gives the story more reach, touches more people’s lives and hopefully makes people safer as we enter the next chapter of the pandemic.