Calibrating Stitched Videos with NVIDIA VRWorks 360 Video SDK

The quality of 360 videos varies widely depending on the resolution of the cameras used to capture them. The need for higher quality captures has given rise to many solutions, ranging from 3D printed rigs for GoPro cameras all the way up to professional custom-built 360 camera systems. One particularly difficult task is to align multiple videos from an array of cameras and stitch them together into a perfectly uniform sphere, removing lens distortion and matching color and tone. In an effort to simplify many of the intricacies of generating video with 360 camera systems, NVIDIA provides the free NVIDIA VRWorks 360 Video SDK, which is being adopted into many production applications supplied by 360 camera makers. A new NVIDIA Developer Blog post introduces 360 video camera calibration and the new features available in the latest version of the SDK.

The NVIDIA VRWorks 360 Video SDK version 1.1 offers a calibration module that determines the configuration parameters of a multi-camera rig. Calibration is designed to work in real-world settings, not just in the lab. It uses the correlation between scenes observed by several cameras to produce an accurate mapping between the camera coordinate system and the world coordinate system. The SDK supports a wide range of cameras and rig configurations, as long as the rig is reasonably compact (smaller than 0.5 meter diameter).

In addition to multi-camera video calibration, version 1.1 of the VRWorks 360 Video SDK includes a number of other improvements.

  • The SDK can accept estimates of camera parameters and rig properties from the user to improve the rate of calibration convergence.
  • Automatic calibration mode: calibration with few or no estimates of rig parameters including camera orientations and lens properties.
  • Nonhomogeneous rig support: different camera resolutions or lenses in a single rig.
  • Support for normal perspective cameras and wide-angle fisheye camera lenses.
  • Built-in rig balancing along the horizon for equatorial ring camera rigs.
  • Built-in cross-correlation-based quality measurement of calibration accuracy.

The video below is an example of using the SDK to generate a 360-degree video (note you can pan the view around with the mouse).

Read more >