360 videos are a powerful and easy way of creating immersive 3D experiences for your academic VR experiments and many other applications. This article gives an intro and discusses the trade-offs between 360 video and computer generated 3D content.
To get a feel for 360 videos, please first take a look at this Youtube video showing a 360 video application of an immersive classroom experience:
You can use your mouse to look around the video or if you have access to an Oculus Quest I would recommend downloading the Youtube app for Oculus and watching that video in the Quest to get a sense of what the experience feels like.
Trade-offs between 360 video and CG Environments
360 video provides some really interesting benefits when compared to traditional computer graphics (CG) environments and I've outlined some of the trade-offs below.
Benefits:
Highly detailed environments such as a classroom can be captured by using a camera. When running on standalone devices such as the Quest this allows for more detailed environments in general since the environment itself does not have to be rendered which for mobile headsets requires a high degree of optimization in order for a smooth experience.
Human interactions can also be displayed very realistically since you are capturing actors
The cost of producing a video of a complex scene is much lower than a traditional CG environment since all objects do not have to be modeled, just filmed. This is especially true of human actors since realistic avatar animations are some of the most labor intensive CG elements.
Computer graphics elements can still be added to a scene to allow for some interactivity - the most common is a menu interface that pops up at a predetermined time with a multiple choice set of answers that determine how the experience proceeds. For example, a participant could be prompted with a math question and then have the option to select the correct answer with their controller from 4 choices that are hovering in front of them. After the selection is made the video proceeds (a correct answer triggers a "correct answer reaction" video and a wrong answer triggers an alternate video indicating the participant has gotten the answer wrong and prompts them to try again.
Limitations:
Participants are not able to move freely through a 360 video environment - they are limited to the perspective of where the camera is placed whether that is seated or standing. If they get up and move while wearing the headset their relative position within the experience will not move. This disconnect between body and perceived experience can occasionally lead to nausea, especially if the camera perspective is moving so it's best to have a stationary camera and participant.
Interactivity is limited as you can't reach out and grab objects within the environment.
Filming presents its own set of challenges as you need a cast, crew and camera which can be difficult with current pandemic restrictions. CG environments can be developed remotely by distributed teams.
Overall you have much more flexibility with the development of CG environments since elements can be added or removed / improved at a later date. With 360 video you would have to go and film an entire new scene if anything needed to be added and you are more or less stuck with what you have filmed for generating the final experience.
For more information on how you can use 360 videos in your simulations, including how you can measure eye tracking data, add areas of interest and more contact sales@worldviz.com
Additional Resources:
We use and recommend the Insta360 Pro II as a great camera option (it currently retails for around $5,000) - it is capable of producing both mono and stereo video in 8K.
This paper from Linnaeus University provides some interesting insights to 360 video techniques in general and specifically how attention can be directed within a 360 video environment.
