The No.1 Website for Pro Audio
Monitoring using 360 video
Old 31st July 2020
  #1
Monitoring using 360 video

Fellow Gearslutz members,

I'm currently studying at UWL in London (Record Production MA), albeit I'm obviously not actually physically attending classes, or even in the country anymore for that matter.

For my final project I'm looking to use 360 video for monitoring purposes. By that I mean playing along to a VR video of me playing when overdubbing additional layers.

This requires getting GoPro 360 footage to play in time on/through the VR spectacles whilst Pro Tools is playing (the footage will be imported into Pro Tools).

So far (when using regular footage) I have achieved this by connecting an external monitor to my Mac, and importing the video into Pro Tools, firstly syncing it to the audio in the session in Premiere, so it starts at the same point on the timeline.)

I do not require spatial immersive sound when overdubbing, and won't even be wearing headphones, as I will reamp all previous takes in the same space through amps and drum trigger hammers.

Here are links that explains the basic ideas in case anyone is interested or requires more details.


https://youtu.be/HpZErq5dKC8

https://www.dropbox.com/s/gxlbo1w1zx...35376.pdf?dl=0

https://1drv.ms/v/s!ArNraO7sMBE3g2_PkRIsyMxz9p4G


My question is, has anyone got any experience with trying something similar? I know most people probably only use and/or import 360 visuals when doing post production, so it's visible when mixing immersive sound. But perhaps someone has thought of this before (probably)...?


My current ideas are:


- Finding a way of syncing a video player with Pro Tools through time code, in this way not requiring the video engine in Pro Tools to operate

- Separately recording when playing along to a 360 video that has been rendered, but if I do this I can't reamp the sounds in the room, which is vital for this experiment. Also this would require syncing takes with Pro Tools afterwards, although this isn't super hard.

- Giving up on the idea and just using separate monitors that represent the past takes and players (myself in this case) in the correct positions and not using VR visuals. (This is more like plan B in case I can't get it to work.)

If I stream from my MAC , whether from Pro Tools or a separate player, the streaming will introduce latency, so the video will not play in time with what I'm hearing in the space, which is not an option. I don't think it would be constant, so it can't be compensated for either I think.



I have access to:

- Pro Tools newest version (Subscription)

- Teacher/Educational Pro Tools 11 (full version)

- Logic Pro X (every time I use it I tend to quit seeng it's so cumbersome when used to Pro Tools, but I could use it. Not saying it's not good just that I'm not used to using it so preferably I'd avoid using it.)

- Reaper (I can buy Reaper in case it's easier to do in there, although I'm also not familiar with it, I just listed it since It's known for being a lot more flexible than Pro Tools)


I realise there are a lot of potential issues I haven't even mentioned here, or considered at all, but I figured this major hurdle needs to be overcome first.


Thanks for thinking with me.


- BL
Old 31st July 2020
  #2
Quick addition, here are some photos to better contextualise the idea if you don't have time to skip through the linked videos/documents. The motivation is to facilitate improved free time communication and interaction between one agent through time:


(I'm not actually very proficient at drums, but I think most people would agree layering an entire recording in free time as on e person doesn't usually result in the tightest sound, admitting musicians better than I would achieve better results of course.)
Attached Thumbnails
Monitoring using 360 video-screenshot-2020-07-31-12.21.54.jpg   Monitoring using 360 video-screenshot-2020-07-31-12.19.43.jpg  
Old 2nd August 2020
  #3
Lives for gear
 

Quote:
Originally Posted by BarryLimestone View Post


- Finding a way of syncing a video player with Pro Tools through time code, in this way not requiring the video engine in Pro Tools to operate
If you have access to a separate Windows machine you can sync Pro Tools to the Facebook Spatial Workstation's 360 video player via timecode over ethernet.

FB360 is free, and this works both with desktop video or an oculus rift.
Sometimes kinda buggy though. Support is via an Facebook group.

Good luck!
Old 2nd August 2020
  #4
Thanks

Quote:
Originally Posted by kosmokrator View Post
If you have access to a separate Windows machine you can sync Pro Tools to the Facebook Spatial Workstation's 360 video player via timecode over ethernet.

FB360 is free, and this works both with desktop video or an oculus rift.
Sometimes kinda buggy though. Support is via an Facebook group.

Good luck!

Thanks. I’ll have a look into that. Perhaps I can borrow a windows pc from a friend, as I have a MacBook. Or I could find a different program that will let me do the same with my MacBook and iMac.
Old 2nd August 2020
  #5
Lives for gear
 

Quote:
Originally Posted by BarryLimestone View Post
Thanks. I’ll have a look into that. Perhaps I can borrow a windows pc from a friend, as I have a MacBook. Or I could find a different program that will let me do the same with my MacBook and iMac.
You can also try if it works with two Macs. I never looked into that as we're only using it for driving an Oculus Rift for 360 video spatial mixing, and the Oculus only works on windows for now.
Old 3rd August 2020
  #6
Quote:
Originally Posted by kosmokrator View Post
You can also try if it works with two Macs. I never looked into that as we're only using it for driving an Oculus Rift for 360 video spatial mixing, and the Oculus only works on windows for now.
I’ll let you know how it pans out!
Old 3rd August 2020
  #7
Gear Maniac
 

Quote:
Originally Posted by kosmokrator View Post
You can also try if it works with two Macs. I never looked into that as we're only using it for driving an Oculus Rift for 360 video spatial mixing, and the Oculus only works on windows for now.
I've been side loading my VR/360 mixes to my Oculus Quest from my Mac for the past few months. For me it's a better solution than syncing a windows machine.
Old 9th August 2020
  #8
Quote:
Originally Posted by Tapwatr View Post
I've been side loading my VR/360 mixes to my Oculus Quest from my Mac for the past few months. For me it's a better solution than syncing a windows machine.

I'll have a look into that as well!
Thanks.
Old 9th August 2020
  #9
I think I might try the HTC Vive, which works with my mac. That means I can sync the video to the daw on one computer without issues... hopefully....


So if I understand correctly @ Tapwatr , you are able to sync your PT session with the playback of the 360 player internally and then preview/watch it through a connected VR headset?
Old 10th August 2020
  #10
Gear Maniac
 

Quote:
Originally Posted by BarryLimestone View Post
I think I might try the HTC Vive, which works with my mac. That means I can sync the video to the daw on one computer without issues... hopefully....


So if I understand correctly @ Tapwatr , you are able to sync your PT session with the playback of the 360 player internally and then preview/watch it through a connected VR headset?
Yes, I use the FB360 video player while editing/mixing. When I feel my mix is close and I want to watch in VR I bounce it, marry it to the video with FB360 Encoder. The last step is to "side load" that file onto the Oculus which is connected to the same Mac my PT system is on.

You have to install and run these two apps to be able to side load to the Oculus Quest. I'm not sure if this method will work on a Vive.

https://sidequestvr.com/

https://www.android.com/filetransfer/
Old 10th August 2020
  #11
Quote:
Originally Posted by Tapwatr View Post
Yes, I use the FB360 video player while editing/mixing. When I feel my mix is close and I want to watch in VR I bounce it, marry it to the video with FB360 Encoder. The last step is to "side load" that file onto the Oculus which is connected to the same Mac my PT system is on.

You have to install and run these two apps to be able to side load to the Oculus Quest. I'm not sure if this method will work on a Vive.

https://sidequestvr.com/

https://www.android.com/filetransfer/
So that way you can't monitor/watch through the VR headset as the session plays? I want to reamp some stuff, and use midi to trigger percussion hammers, as I watch the footage.. haha.

I will experiment with this, but using time code to sync two machines might be the best way.
Old 10th August 2020
  #12
Gear Maniac
 

Quote:
Originally Posted by BarryLimestone View Post
So that way you can't monitor/watch through the VR headset as the session plays? I want to reamp some stuff, and use midi to trigger percussion hammers, as I watch the footage.. haha.

I will experiment with this, but using time code to sync two machines might be the best way.
No, I can't watch as session plays, it's a bit over rated When I started in VR I use to sync to a Windows machine but I prefer this method now.
Old 11th August 2020
  #13
Lives for gear
 

Quote:
Originally Posted by Tapwatr View Post
No, I can't watch as session plays, it's a bit over rated When I started in VR I use to sync to a Windows machine but I prefer this method now.
Well, I don't agree there. We do side-loading after the fact too, but the perspective is so off when not looking through the goggles that we need to be able to check through them in realtime.

Mainly depends if you're mixing for facebook or for goggles distribution mainly though.

Best of course is to have both.
Old 15th August 2020
  #14
Gear Maniac
 

Quote:
Originally Posted by kosmokrator View Post
Well, I don't agree there. We do side-loading after the fact too, but the perspective is so off when not looking through the goggles that we need to be able to check through them in realtime.

Mainly depends if you're mixing for facebook or for goggles distribution mainly though.

Best of course is to have both.
Which spatial mixing software are you using? The reason I don't mind not having the HMD is with the FB360 I'm pretty good at getting the spatialization right so it's just minor tweaks after watching it in VR.
Old 16th August 2020
  #15
Quote:
Originally Posted by Tapwatr View Post
No, I can't watch as session plays, it's a bit over rated When I started in VR I use to sync to a Windows machine but I prefer this method now.
I don’t want to use it for mixing though, it’s for when you’re overdubbing yourself so you can play along to your previous take standing in the same space (with preserved perspective through 360) whilst overdubbing. I know it seems like a strange thing to do, but this is my project’s theme/subject haha. I’ll guess I’ll be syncing to a windows laptop...
Old 16th August 2020
  #16
Gear Maniac
 

Quote:
Originally Posted by BarryLimestone View Post
I don’t want to use it for mixing though, it’s for when you’re overdubbing yourself so you can play along to your previous take standing in the same space (with preserved perspective through 360) whilst overdubbing. I know it seems like a strange thing to do, but this is my project’s theme/subject haha. I’ll guess I’ll be syncing to a windows laptop...
Well, in that case, I guess you need a PC.
📝 Reply

Similar Threads

Thread / Thread Starter Replies / Views Last Post
replies: 52 views: 14476
Avatar for BenJah
BenJah 25th September 2008
replies: 218 views: 54675
Avatar for jimdrake
jimdrake 20th December 2012
replies: 1296 views: 151203
Avatar for heraldo_jones
heraldo_jones 1st February 2016
replies: 0 views: 4332
Avatar for golden beers
golden beers 29th May 2015
Topic:
Post Reply

Welcome to the Gearslutz Pro Audio Community!

Registration benefits include:
  • The ability to reply to and create new discussions
  • Access to members-only giveaways & competitions
  • Interact with VIP industry experts in our guest Q&As
  • Access to members-only sub forum discussions
  • Access to members-only Chat Room
  • Get INSTANT ACCESS to the world's best private pro audio Classifieds for only USD $20/year
  • Promote your eBay auctions and Reverb.com listings for free
  • Remove this message!
You need an account to post a reply. Create a username and password below and an account will be created and your post entered.


 
 
Slide to join now Processing…
🖨️ Show Printable Version
✉️ Email this Page
🔍 Search thread
🎙️ View mentioned gear
Forum Jump
Forum Jump