by Dan Neafus, IMERSA co-founder; Manager, Gates Planetarium
Dan Neafus and his “Language of Fulldome” live presentation/demo have been in demand for several years at conferences and film festivals. Neafus points out that the unique, immersive properties of the medium and their effect on the viewer call for a unique approach to storytelling that departs in many instances from traditional filmmaking techniques. The presentations have grown into a full-fledged written treatise on the subject, of which this is an excerpt. Dan cites as one of his foremost inspirations in developing these concepts filmmaker Prof. Ben Shedd of the School of Art, Design and Media at Nanyang Technological University in Singapore, particularly Shedd’s “Exploding the Frame” research.
Modern storytellers and technicians are inventing a new language for immersive cinema which may be expressed in many ways. This evolving experiential language is a synthesis of jargon and techniques from many disciplines, each contributing its own idiosyncrasies to our conversations.
Within immersive cinema we must recognize that as you move the camera, you are moving the audience, physically and emotionally. The camera is the viewer and our virtual camera represents not just the audience perspective, it is the viewer’s reality.
Language of the sphere
An immersive screen may be thought of as a bubble ship, enclosing an audience. When describing the attributes of a shot we are actually describing what is happening to the audience while traveling within this bubble, consider that the audience is the camera.
Shooting
Linguistic challenges persist as we begin to define techniques for choreographing immersive cinema. The term shooting is ambiguous because we are no longer aiming and shooting at things. The recording or rendering may be initiated by simply clicking a mouse button. Our cameras are often virtual, or it may be some extremely wide angle, omnicamera designed to acquire every sight and sound in a spherical scene, again the recording may be triggered remotely with a finger touch.
We can redefine our language of shots by modifying some conventional shooting descriptions. Currently accepted camera lingo uses the perimeter of the rectangular frame as a reference and describes the apparent movement of subjects within the viewfinder frame. Scenes are composed in relation to this frame, by moving the camera: vertically, horizontally, far, close, or by having the subject(s) themselves move, or by manipulating the optical mechanical lens focus and magnification.
To define a fulldome sequence, a director needs to describe audience motion, while disregarding the viewfinder frame. They must use techniques to illustrate exactly what the audience sees around them, and how they are moving in relation to the subjects. These audience motions are similar to conventional camera moves rather than lens movement, and require a more appropriate, audience-centric terminology to describe them.
Zooms
When an audience moves away from a subject more of the surroundings are revealed around it creating a wider field of view. This is not done by optical means, as is typical in zoom lens shot. It is done exclusively by moving the audience viewpoint from one location to another. Thus a wide shot is a viewpoint from very far away from the subject, taking in most of the scenery around the subject. A long shot or establishing shot is a vantage point that is far from the subject. A medium shot is closer to the subject and a close up is an audience view near the subject. A tight shot is acquired when the audience feels very close to the subject.
Experiential precedent
Many sequences from immersive cinema need to be experienced to be understood. Real time data navigation software has simplified the choreography of complex scenes by allowing a piloting director to use a flight control device to rehearse a flight path sequence through a model and then save out representative frames as a storyboard, illustrating important moments within the sequence.
Using this real time technique, a director can actually rehearse each sequence in person. When rendered these unique experiences become part of a catalogue of experiential precedents that all film makers can reference. Audiences will be able to experience these moving scenes in person and become familiarized with these new techniques, adding to their visual and experiential literacy.
The majority of shooting conventions are useless when creating complex immersive scenes where an audience moves among other moving objects. These audience motions help to tell the story and often create interaction with the scene’s content. For example our audience feels like they are shrinking as they pass through the eye of a needle during the fulldome film NANOCAM by Roberto Girón.
In the film ALL WE ARE by Carter Emmart, illustrating an adaptation of the Ray and Charles Eames movie POWERS OF TEN, the audience slowly spirals down toward objects on a tabletop. They feel like they are shrinking, as objects appear larger and larger in the foreground. Large objects loom in the background while emphasizing ever smaller objects appearing one by one in the foreground.
Experiential Language
Immersive cinema is experiential cinema, engaging the audience in a shared voyage.
Discovering places otherwise impossible to visit, the audience transcends physical limitations of time & distance, shrinking down to witness tiny details of atoms, or perhaps travelling back in time to explore our primordial origins. Immersive cinema is a shared experiential voyage within a common sphere of perception.
Science visualization artist Robert Patterson endeavors to disseminate his team’s earth and space science by flying audiences through the data assembled by his team. This type of reality based immersion helps scientists to create accurate virtual models directly from their own data. Once visualized, a researcher can navigate and share their model of reality with colleagues or the public.
Scientific visualization is utilized in many disciplines of research, from archeology to astrophysics. Virtual voyages take on special meaning when exploring scientific data sets acquired through cameras, telescopes, microscopes or satellites. Features that are collected digitally may remain unseen in their raw digital format, until they are rendered into reality through scientific visualization and virtual immersion. When the dynamics of time and motion are added into the equation, participants can truly experience being within a phenomenon.
Language of the edit
Today’s editing language is quite sophisticated and its comprehension is common among the modern movie audience. These conventional edits work well for small screens but may be quite disorienting when used in the immersive environment. Some fulldome directors choose instead to choreograph an uninterrupted camera path, arguing that the best edit is no observable transition at all.
Continuous scenes
To avoid confusing an audience and help them to follow the movement of the story, some directors try to design uninterrupted sequences into long continuous shots throughout the story. This immersive editing technique guides an audience effortlessly throughout a film. Any unavoidable transitions between scenes are carefully planned for the giant screen, resulting in a seamless voyage throughout the entire story.
Orbiting
Another traditional film editing technique sequences reverse angle shots by quickly cutting between several points of view. A preferred approach for immersive cinema takes a completely different approach by slowly orbiting the camera around the subject, providing the audience with views from different sides of the action. This is much less confusing to the viewer than multiple hard cuts from opposing sides.
The fulldome film WE ARE ASTRONOMERS by Max Crow effectively uses this orbiting technique and a corresponding subsonic sound effect, to reveal complex phenomenon as the audience orbits two impacting particles in the Large Hadron Collider. This orbiting technique preserves context throughout the scene and avoids confusing the audience.
Depth of field and racking focus techniques work well for moving pictures and have been adapted for immersive cinema to emphasize a subject through manipulation of focus and detail. Reveal shots also work well in immersive cinema. Reveals may have the subject appear out of darkness or appear larger and larger as they approach from obscurity. It is also quite effective to reveal subjects by having them move into view from the bottom edge of the screen. Peter Popp’s film REALM OF LIGHT using dark, rocky asteroids to frame the next scene and then drift out of view to fully expose a giant red star looming in the background.
As a common language for immersive cinema is translated and adopted, we should become more proficient in communications with our peers and will give our audiences more meaningful immersive experiences.
Fulldome theaters are not only hemispheres of experience, they are spheres of influence. Audiences and producers are becoming more and more informed, learning what can be done, and what to expect from an immersive film. Yet immersive cinema remains a naïve format offering unprecedented access and creative opportunities to producers.
Muriel Rukeyser’s observation that “the universe is made of stories not stars,” reminds us of the timeless importance of our stories, inspiring us to become more fluent in our language of immersive storytelling and to connect with our audiences by speaking to their hearts. Immersive cinema offers unprecedented access and creative opportunities to producers. • • •
