360: in the Field and the Editing Room

Three months prior to the purchase of 360 equipment, it was necessary for our nonprofit video team to clarify our potential predominant applications of the medium. This helped us foresee to some extent what types of environments we’d be filming and the challenges those environments might pose. This clarification provided the parameters within which we conducted our equipment research and ultimately our equipment purchases. Equipment intimacy occurred via four months of application experimentation followed by a five-month Master’s level 360 videography course that offered an in-depth focus on postproduction 360 editing in Adobe Premiere Pro, After Effects and Mettle Sky Box. Research, application, skill acquisition, and a deeper understanding of the differences between 2D and 360 storytelling prepared us for challenges in the field for outside-the-box storytelling and postproduction room salvations.

Three months of comparative research went into choosing the just-right equipment but not before we defined our primary applications and environments in which these applications would take place. Our primary filming environments would be considered hostile for most camera’s on the market: wet, dank, muddy and mucky, one to thirty feet deep in salt and freshwater, wild animals, forest floors, Everglades thicket and alligators. We chose our equipment based on these environments and the challenges that these environments presented. Our final choice landed on the Nikon KeyMission 360 4K, a durable, waterproof to 100 feet, 2” x 2” x 1”, black camera with inexpensive interchangeable and replaceable lenses (incase one gets knocked off or scratched). An additional big winner with this camera was the self-stitching firmware. At the end of a shoot, there’d be no wait to review footage. We simply remove the micro card, insert it into a reader, open it in Premiere Pro, and start editing. At full resolution 4K, 360 video files can be extremely dense and so require a lot of processing time. The two-lens capture typically has to be imported into, stitched by, then exported from proprietary software. This consumes tons of time and the process is not without glitches. Technological forward trajectory seems headed towards firmware stitching and not likely to regress, the self-stitching combined with the camera’s durability ensured the most efficient capture of our stories as we envisioned them.

Familiarity with 2D filming meant we possessed many of the necessary skills to produce a 2D film, and some of these skills did transfer to our 360 applications, but by trial and error we now understand that 360 presents some pretty unique challenges. Our first big realization was that there was no place to hide. We were either in the shot or we were not. If in it, our presence had to add to the story or at the very least not detract from it. If we chose not to be in the capture, then the big question became, where could we blend in or hide from sight. In some of our shoot locations, there was nowhere to hide. In 2D filming, we hide behind the camera. The viewer only sees what’s placed in front of the one lens. Not so in spherical.

The second unique challenge that presented in our experimentation was lighting. Lighting can only by subterfuge be introduced into a 360 shot, because remember, anything within the visual sphere of the camera’s lenses (it’s 360) is captured. Any lighting rig, if not cleverly hidden or immersed will be visible. Night time in low lit areas turned out to be the most difficult capture for us, but we found that reflective surfaces and small well placed, out-of-sight LED flashlights can do a lot for increasing the ambient light of a dark space, either suspended from the tripod to bounce off a small reflective disc that can be cloned out in After Effects. Some creative solutions ranged from diverting diffused lights from overhead tree branches, or in the branches of shrubs. A little extra creative genius is required when it comes to innocuous staging, and the need varies set to set and is heavily reliant on adaptable gear in the gear bag, and we cannot stress enough the importance of a pre-shoot visit to the shoot site about the same time of day the shoot is planned for. It’ll help determine any equipment necessary for a successful shoot.

We discovered, too, that 360 lenses were themselves quite the challenge. Most of the micro 360 cameras we experimented with: the Samsung Gear 360, the KodakPro 360, the Nikon KeyMission 360, the 360 Fly, and the GoPro 360 rigs, shoot with extremely wide angle lenses for maximum field of capture (near lens to lens). These lenses do not have adjustable focal length, exposure, or irises. In order to get close details and on any character or object, that character or object must be up close in the camera’s face. One might think that this is any easy solution. Just place the camera closer, but this is not as easily done as it is in 2D, especially when you also have to make sure the sun is not blinding either lens and the camera position is filming from the best perspective and the peripherals and the tertiary areas are contributive but not so overly contributive as to distract from the point of desired focus. Only by carefully considering all of the aforementioned issues can we contemplate the placement of our camera so as to get a more detailed close-up capture while maintaining the integrity of the Spherical experience.

Power was a simple solve. A 2-hour, 5 volt DC power-pack, ten spare batteries for each camera and light, and twenty double A batteries for the Zoom H6 and our two shotgun mics made sure that power would not be an issue for us. All of the these power supplies were stored in a small dry-bag. However, we discovered with the loss of an entire day’s shoot that the Nikon KeyMission 360 camera LEDs indicate active recording while plugged into one of the 2-hour power packs, when in fact the camera is not actually recording. Each camera uses one camera battery every hour. Ten per camera meant we were covered, so long as we remembered to charge all batteries before each shoot. This also meant that we had plenty of power even without the power-packs. Our hidden one double A powered mic plugged into the Zoom H6 powered by four double As lasted for two days before the batteries needed replacing. Power did not present a challenge in the field. Data collection and storage posed as little a challenge as power supply. Sixteen ten-speed 32GB micro cards could store up to eight hours of full 4K resolution data. To be on the safe side, we stored all the micro cards, used and unused, in a waterproof snap case.

Audio proved as well to be quite the challenge. While the Nikon KeyMission 360 has a surprisingly powerful mic, like most other 360 cameras, ambient and signal sounds are captured indiscriminately. Separating out a specific signal from high ambient locations, requires some serious audio skills in After Effects or it requires some up front forethought about what audio we want to capture, how that capture can be accomplished, and what equipment we need to capture it. In 360, whatever equipment we go with will either be openly in scene or hidden with great skill. So what does the situation call for? In one scene along a busy and extremely loud causeway bridge, we mounted a shotgun mic on the outside of the highway guardrail, over the water, and aimed it up at our interviewee. We decided that we could clone out the protruding top of the mic impossible to hide, using Adobe After Effects. This solution was problematic though and proved unworkable. Every time the interviewee moved along the guardrail and cast his net, he dislodged the mic. We almost lost it in the water once. Another solution was to not get direct face shots and add a conversation voice-over where visually plausible. The lavaliere mic was not possible since it too was compromised and dislodged by the interviewee’s actions. In most cases, the highway traffic won and we finally decided to to do a creative voice over.

Our innovative 360 solutions met most field challenges with great success, even a few designed to push the limits of our equipment and application possibilities. Most innovative accomplishments were viable only with the possession or acquisition of some fundamental postproduction skills: from the desperate act of layering a weak signal on top of itself, hoping it would at the very least document momentous effort; to color correction and masking; to the fine art of splice, dice, mix, and mingle, only because we possessed some fundamental postproduction skills did we pull a few rabbits out of hats. While we may not have found uncomplicated solutions or easy work-arounds in every case and some footage and audio still ended up unusable, our editing skills made most of our efforts payoff with some successful and interesting take-aways.

Like 2D or novels and short stories, there has to be a timeline that draws, pushes, and pulls the reader/viewer/end-user through the experience to the end, but unlike 2D, we can’t easily jump from perspectives and spaces (scene to scene). Quick glimpses don’t work in 360 the way they do in 2D. If the user isn’t given a moment to settle and look around briefly, what’s the point of 360? Unlike 2D viewing, 360 jumps can and often do create spatial disorientation. Change perspectives but not scene or setting, the jar is less traumatic, but jump from one extreme to the other: the top of a highway bridge to underwater surrounded by a school of fish… It’s disorienting. It disturbs the senses and dislodges us from any connection we’ve managed to create for the users thus far. All of these are acceptable if done with intention to cement an experience within the experience, if not, it’s just a glitch. How do we mitigate this spatial displacement sensation or use it as an effective tool to sculpt specific impacts? When creating our 360 timeline, we must consider this spatial distortion experienced by the end user when we jump them from perspective to perspective and space to space.

The 360 threshold medium presents us with an endless array of potential applications and so an endless array of application challenges. With the right equipment chosen to meet specific strenuous project demands and with the postproduction skills to mitigate any field failings, most of our efforts prove repeatedly fruitful. Our 360 stories, from great camera placement to innovative methods of capturing scene and sound, can bring our end user into the life experiences of the characters, animate and inanimate, that our stories present.