Hi! — I made this (very low quality) video with the following pipeline that has 360 video paired with ambisonic B audio and ensures that audio is all coming from the proper locations within the space.
If you watch this in VR, everything is properly spatialized and shows up correctly in the room with sound coming from the proper locations!
Record audio and video with your camera and ambisonic microphone. I’m using this camera and this mic.
Make sure the camera and mic are aligned. I found for my camera / mic set up, that means orienting them like this:
Press enter or click to view image in full size
both the camera and the microphone facing forwards
Use an SD card reader to pull the data off of both the mic and the camera. Don’t bother with any of the apps they’re trying to give to you, they are super finicky.
Open audio in audacity to do your basic amplification, do noise reduction, etc.. You can edit this just like a normal audio file. TIP: don’t hit normalize because then it’ll boost the different tracks a different amount to make them all equal. This is like a corner case of “normalize stereo channels independently” for ambisonics (four tracks) that isn’t considered yet I think.
Export with “enable advanced mixing” to get your 4 channel audio output.
Open the raw video and audacity exported audio in DaVinci resolve. Cut things up and make it all edited / trimmed. You’ll notice I did none of this above, other than aligning on the snaps at the beginning of the track.
[DaVinci tip 1] Make sure to remove audio track from camera once it is lined up, and to delete the track, not just the clip, so that your four audio tracks are the only audio tracks.
[DaVinci tip 2] Make sure that resolution on davinci is 3840x1920.
[DaVinci tip 3] Pull up distribution tab, make sure you select “All Timelines” for audio
[DaVinci tip 4] Add to render cue, and render
Take the exported clip and run:
Get Brian Ellis’s stories in your inbox
Join Medium for free to get updates from this writer.
Pop the four resulting wav files (in order) into a new audacity file so you have a single file with four tracks.
Export that again with advanced mixing options to combine them into one, called “finalout.wav” (these last two steps could probably be fixed with fancy ffmpeg commands, but I’m don’t know how to do this)
Open up the fb360 encoder.app (downloaded from here) and select the finalout.wav as your audio file with b-format (1st order ambiX) and the video layout as monoscope.
Export that stuff to youtube or FB files and upload as you see fit!
To view the videos properly in VR, I’ve been using Youtube VR’s app in the Oculus Quest.
To close, please enjoy this 360 sound bath I recorded using this pipeline. This has the additional step of layering the audio and video. This was done using very conventional techniques layering each of the video tracks in DaVinci and mapping tracks together in Audacity, and a little bit of color correction magic. Hope you enjoy!
