krpano 1.19-pr16 (build 2018-04-04)

WebVR Plugin webvr.js

  • krpano WebVR / MobileVR support.
  • Experience any 360 panoramic image or video (normal or stereoscopic) in 3D Virtual Reality mode directly inside the browser.
  • Either by using a dedicated Head Mounted Display like the Oculus Rift or the HTC Vive or by using mobile devices with VR-Headsets like Google Cardboard, Gear VR or similar.
  • The plugin uses either the new (experimental) WebVR Browser API or for mobile devices/browsers without WebVR support, the accelerometer and gyroscope sensors.

Topics

Syntax

<plugin name="WebVR" devices="html5" keep="true"
        url="webvr.js"
        zoom="1.0"
        friction="0.0"
        worldscale="1.0"
        mousespeed="0.00125"
        oversampling="1.0"
        multireslock="true"
        headtracking="true"
        fullscreen_mirroring="false"
        mobilevr_support="true"
        mobilevr_ipd="63.5"
        mobilevr_screensize="auto"
        mobilevr_lens_overlap="1.0"
        mobilevr_lens_fov="96"
        mobilevr_lens_dist="0.6"
        mobilevr_lens_dist2="1|0|0|0"
        mobilevr_lens_ca="0.0"
        mobilevr_lens_vign="100"
        mobilevr_orientationlock="true"
        mobilevr_wakelock="true"
        mobilevr_sensor_mode="3"
        mobilevr_autocalibration="false"
        mobilevr_touch_support="false"
        mobilevr_fake_support="false"
        mobilevr_database_url="..."
        vr_cursor=""
        vr_cursor_enabled="true"
        vr_cursor_onover=""
        vr_cursor_onout=""
        onavailable=""
        onunavailable=""
        onunknowndevice=""
        onentervr=""
        onexitvr=""
        />

Plugin Attributes

Attribute nameTypeDefault value
zoomNumber1.0
  • Add a custom zoom to the VR view.
  • Should be used very carefully!
Attribute nameTypeDefault value
frictionNumber0.0
  • A value between 0.0 and 0.999
  • Add a fricition to reduce the gyro responsive speed.
  • Should be only used for higher zoom values, but never for normal viewing!
Attribute nameTypeDefault value
worldscaleNumber1.0
  • The 'VR world scaling' - this value defines how near or far way hotspots are appearing.
  • The panoramic image itself or the sizes of the hotspots will not change.
  • With smaller values (0.1 to 1.0) everything will look smaller and more near to the face and with larger values (1.0 and up) more far way.
  • When using stereoscopic panoramic images it can make sense trying to adjust this value to get a better matching between the panoramic image and the hotspots (and the vr_cursor). This depends how the stereoscopic image was shoot (stereo camera-distance).
Attribute nameTypeDefault value
mousespeedNumber0.00125
  • When being in WebVR fullscreen mode on desktop computers, it's possible to rotate the view horizontally by moving the mouse left and right (additionally to the view rotation by the head-movement).
  • The mousespeed setting controls the speed of that mouse movement.
    Set the value to 0.0 to disable the mouse movement.
Attribute nameTypeDefault value
oversamplingNumber1.0
  • The oversampling factor for the rendering.
  • Higher values can increase the quality, but require more GPU processing power.
  • Lower values will lower the quality and require fewer GPU processing power.
Attribute nameTypeDefault value
headtrackingBooleantrue
  • By setting headtracking to false, it would be possible to disable the headset-movement-tracking and control the viewing direction by yourself by changing the view settings.
  • Warning - disabling this settings is NOT recommended!
  • Changing the viewing direction in VR different to the users head movement is a bad experience. And on systems with 're-projection' (HTC Vice, Oculus Rift) this can cause graphical errors, because these systems doesn't assume that the view will change different to the head-movement.
Attribute nameTypeDefault value
fullscreen_mirroringBooleantrue
  • For desktop WebVR usage.
  • Switch to fullscreen mode when mirroring the WebVR output.
Attribute nameTypeDefault value
multireslockBooleantrue
  • When this setting is enabled and when using a multires pano, then the multiresolution loading will be 'locked' to one specific resolution level.
  • That locked level will be fully loaded when entering the VR mode or when changing the pano in VR mode.
  • The fully 'pre-loaded' image resolution level avoid loading interrupts during the viewing. This is important because especially in VR mode any interrupts or frame drops are strongly noticeable.
  • Note - that full preloading can cause a short performance drop during the initial loading of the pano, but therefore there will be no interrupts during the viewing itself.
Attribute nameTypeDefault value
mobilevr_supportBooleantrue
  • Enable or disable the MobileVR support.
  • This setting allows to use VR on any mobile device (also without WebVR API support).
Attribute nameTypeDefault value
mobilevr_ipdNumber63.5
  • Set the interpupillary distance (IPD) in millimeters (mm).
  • This will be used in MobileVR mode to achieve the correct stereoscopic 3D presentation for the user.
Attribute nameTypeDefault value
mobilevr_screensizeString"auto"
  • Set a custom diagonal device screen size in inch.
  • This information is necessary to be able calculate the correct 3D presentation.
  • When set to auto (the default):
    • In this case the plugin tries to detect the device itself to get its screensize.
    • On Android the device detection works by checking the user-agent of the browser. The plugin has an intern list of several devices and their screen sizes.
      If you want your device on this list, just send a short mail including the user-agent and the screen size of your device.
    • When detecting the screen size isn't possible, the plugin assumes a screen size of 5.0 inch and sends the onunknowndevice event.
Attribute nameTypeDefault value
mobilevr_lens_overlap
mobilevr_lens_fov
mobilevr_lens_dist
mobilevr_lens_dist2
mobilevr_lens_ca
mobilevr_lens_vign
 Number
Number
Number
String
Number
Number		 1.0
96
0.6
"1|0|0|0"
0.0
100
  • Set the lens parameters for the MobileVR mode:
    • mobilevr_lens_overlap
      • A scaling factor for lens view overlapping. The value depends on the distance between the lenses and the shape/design of the lenses.
    • mobilevr_lens_fov
      • The vertical field-of-view in degrees.
      • The resulting field-of-view will depend on the screen-size and the lenses.
    • mobilevr_lens_dist
      • The strongness of the lens distortion.
      • Values: 0.0 to 5.0, 0.0=no distortion.
      • This distortion will be rendered by using the krpano internal fisheye view distortion in a one-pass rendering step.
      • There will be no degeneration of the image quality when using this distortion. That means this distortion should be used as first step.
    • mobilevr_lens_dist2
      • This is second-pass post-processing lens distortion step.
      • It can be used to apply an additional distortion when using the mobilevr_lens_dist setting is not enough.
      • When the distortion parameters are causing a zoom, the image quality can slightly degenerate due the scaling and interpolation.
      • There are 4 distortion parameters:
        mobilevr_lens_dist2="k1|k2|k3|k4"
        which were used in this lens distortion model: (r = distance from the lens center)
        r = r * (1.0 / k1) * (1.0 + k2*r2 + k3*r4 + k4*r6)
      • This distortion requires additional GPU processing power and can reduce the frame-rate. When using the default values "1|0|0|0" this step will be skipped.
    • mobilevr_lens_ca
      • Chromatic aberration correction - reduce color fringes.
      • Values: -1.0 to +1.0, 0.0=off.
      • The red and blue color channels will be scaled.
      • The CA correction requires additional GPU processing power and can reduce the frame-rate. When using the default value 0.0 this step will be skipped.
    • mobilevr_lens_vign
      • Add a vignette shading to hide the lens edges or unwanted areas.
      • Values: 0 to 100, 100=off.
      • The vignette will be only added when using the mobilevr_lens_dist2 setting.
  • The default values should are a good fitting for the original Google Cardboard lenses.
  • Lens settings for some VR headsets:
    VR Headsetoverlapfovdistdist2cavign
    Google Cardboard (A)
    (Original design with 60mm ILD* and bi-convex lens)    		 1.1961.001|0|0|00.0100
    Google Cardboard (B)
    (Custom design with 66mm ILD* and plane-convex lens)    		 1.0960.61|0|0|00.0100
    Samsung GearVR 1.01120.951|0|0|00.090100
    HOMiDO 1.01011.101|0|0|00.075100
    Zeiss VR ONE 1.0109.90.001.139|0.093|0.018|0.2070.09035
    ColorCross VR 1.0700.651|0|0|00.0100
    * ILD = Inter-Lens-Distance
Attribute nameTypeDefault value
mobilevr_orientationlockBooleantrue
  • Automatically lock the screen / device orientation to landscape.
  • This works only when the browsers is supporting an API for orientation locking (currently only the latest versions of Android Chrome and Android Firefox).
Attribute nameTypeDefault value
mobilevr_wakelockBooleantrue
  • Wakelock - try to keep the mobile device awaken and prevent that the display switches to sleep / standby mode. This is necessary because during the VR viewing typically no user touches on the display (which would prevent sleeping) are happening.
  • Currently the wake-locking is not a browser feature, currently it is only an ugly hack. The current hack might be not working or could cause problems in newer/updated browser version, therefore it's optionally possible to disable that feature.
Attribute nameTypeDefault value
mobilevr_sensorint1
  • Define which browser event should be used for the device movement tracking:
    • 0 = the deviceorientation event
      • Here the sensor fusion will be done by the browser itself.
      • The data from this event is unfortunately buggy or just bad (slow, inaccurate, wrong) in many Android devices and Android browsers.
    • 1 = the devicemotion event (default)
      • Here the sensor fusion will be done by krpano.
      • The browser provides the raw data from the acceleration and the gyroscope sensors and krpano combines them for getting the final device rotation:
        • The gyroscope data is very quick and accurate but the gyroscope sensor measures only relative rotations and therefore it drifts away from the real physical orientation over time.
        • To compensate that drift, the acceleration sensor is used. That sensor measures the gravity acceleration of the earth and that can be used as absolute reference.
        • But the acceleration data is very noisy and it can be only used for detection 'tilt' rotations (tilt left, right, up or down) - rotations around the device itself can't detected by acceleration, for this the gyroscope data is required.
        • So the data from both sensor will be combined / fused. The acceleration data will be low-pass filtered and used as slow 'stabilization' and the gyroscope data for quick and accurate movements in all directions.
  • Using the devicemotion event (setting mobilevr_sensor=1) is the default and there should be normally no need or benefit to use the deviceorientation event, except maybe for some extreme browser bugs...
Attribute nameTypeDefault value
mobilevr_sensor_modeint3
  • The frame-rendering and the 'sensor-data-events' are happening in different time intervals / rates (depending on the system and browser), so it is necessary to evaluate and interpolate/extrapolate the sensor data somehow to get a smooth and also fast and responsive movement.
  • With this setting, different modes for this process can be selected.
  • Available modes:
    • 0 = Directly use the latest available sensor data. No interpolation or extrapolation. Depending on the sensor-time-intervals of the browser the movement can be either jerky or smooth.
    • 1 = Smoothly interpolate between the latest available sensor data. This will give a very smooth but delayed movement.
    • 2 = Forecast the device rotations and then interpolate between the sensor data.
    • 3 = Extrapolate the latest available sensor data to the current frame time. This will give a fast responsive and smooth movement, but there can be a jerking when the extrapolated/predicted data and the real movement don't match.
    • 4 = Forecast the device rotation to the current frame time. This will give a fast responsive and smooth movement, but there can be a jerking when the extrapolated/predicted data and the real movement don't match.
    • 5 = Forecast the device rotation and extrapolate the sensor data from the latest event to the current frame time.
    • 6 = Forecast the device rotation one frame ahead.
    • 7 = Forecast the device rotation two frames ahead.
  • Note - these settings would need to be rated inside a VR-HMD (head-mounted-display)! E.g. mode=1 looks good and super smooth on the screen, but inside a HMD the delayed movement can be unpleasing. Other modes might look jerking on the screen, but good inside the HMD.
    Additionally it depends on the sensor-rate of the browser, how good each mode will look. A good compromise for different devices are the modes 3 and 4.
Attribute nameTypeDefault value
mobilevr_autocalibrationBooleanfalse
  • Enable an automatic gyroscope calibration.
  • Some devices have uncalibrated sensors and there this automatic calibration can help to avoid an unintended gyroscope movement / drifting.
  • The automatic self-calibration will be done every time when the device will be hold very still, e.g. either by holding the device very steady or by laying the device on a stable table.
Attribute nameTypeDefault value
mobilevr_touch_supportBooleanfalse
  • Enable touch support for rotating / adjusting the horizontal offset.
  • This setting is disabled by default because some VR headsets are touching the screen and sometimes causing wrong input.
Attribute nameTypeDefault value
mobilevr_fake_supportBooleanfalse
  • Enable a fake MobileVR support for tablet and desktop devices.
  • Note - That should be only used for internal testing or for demonstrating the possibilities!
Attribute nameTypeDefault value
mobilevr_database_urlString
Attribute nameTypeDefault value
vr_cursorString
  • Use a hotspot as VR cursor.
  • That cursor hotspot will be displayed automatically above all other hotspots at the center of the screen and act as cursor / pointer. The styling of the hotspot itself can be controlled with the normal hotspot attributes.
  • The stereoscopic 3D depth of the cursor will be automatically adjusted to match the 3D depth of other hotspot when hovering them. This makes it eaiser to focus hotspots with different depths.
  • When the cursor will be over other hotspots, the onover and onout events of that hotspot itself and additionally also the vr_cursor_onover, vr_cursor_onout events will be called. This could be used to implement an automatic clicking when hovering a hotspots for some time.
  • Syntax: 
    vr_cursor="hotspot[name]"
    • name - The name of the hotspot.
    • The hotspot itself should be invisible (visible=false) on startup.
Attribute nameTypeDefault value
vr_cursor_enabledBooleantrue
  • Enable or disable the VR cursor.
  • When disabled the cursor will not trigger onover / vr_cursor_onover events.
Attribute nameTypeDefault value
vr_cursor_onover
vr_cursor_onout
 Action Event
Action Event
  • These events will be called when the vr_cursor will move over and out of others hotspots.
  • The 'scope' of these events will be one of the hovered hotspots. That means these events act like the onover, onout events of the hotspot itself.

Plugin State Attributes (read only)

Attribute nameTypeDefault value
isavailableBooleanfalse
  • Is VR support available?
  • This variable will be first set to its correctly value after the onavailable or onunavailable events.
Attribute nameTypeDefault value
isenabledBooleanfalse
  • Is the VR mode currently enabled?
Attribute nameTypeDefault value
iswebvrBooleanfalse
  • Is the WebVR API available / supported by the browser?
Attribute nameTypeDefault value
ismobilevrBooleanfalse
  • When there is no WebVR API support available, but the device itself is a mobile device with working gyroscope sensor support? ⇒ MobileVR support.
Attribute nameTypeDefault value
isgearvrBooleanfalse
  • Is the browser a GearVR browser (e.g. like the 'Oculus Browser' or the 'Samsung Internet' browser).
Attribute nameTypeDefault value
isfakeBooleanfalse
  • Is in MobileVR fake mode.
Attribute nameTypeDefault value
havesettingsBooleanfalse
Attribute nameTypeDefault value
devicenameString
  • Get the name of the WebVR HMD device or the name of the detected mobile device.
Attribute nameTypeDefault value
devicesizeNumber0.0
  • Get the physical diagonal screen size of the device in inch.
  • The value will be 0.0 when the size will be unknown.

Plugin Events

Attribute nameTypeDefault value
onavailableAction Event
  • This event will be called when either WebVR or MobileVR support is available.
  • Could be used to show a 'Enter VR mode' button. This button could call the enterVR() action to enter it.
Attribute nameTypeDefault value
onunavailableAction Event
  • This event will be called when there is no VR support available.
Attribute nameTypeDefault value
onunknowndeviceAction Event
  • This event will be called when the mobile device and its screen size can't be detected.
  • In this case it might be possible to show a menu or setup screen and ask the user there about the screen size of his device.
Attribute nameTypeDefault value
onentervrAction Event
  • This event will be called when switching into VR mode.
  • Could be used for setting up a custom VR layout - e.g. like showing an 'Exit VR' button and hiding the UI elements that are unnecessary or interfering with VR mode.
Attribute nameTypeDefault value
onexitvrAction Event
  • This event will be called when exiting from VR mode.


Additionally to the direct plugin events, the plugin also dispatches these krpano events: 
<events webvr_onavailable=""
        webvr_onunavailable=""
        webvr_onunknowndevice=""
        webvr_onentervr=""
        webvr_onexitvr=""
        />

Plugin Actions

enterVR()
  • Switch into the VR fullscreen mode.

exitVR()
  • Exit the VR mode.

toggleVR()
  • Toggle the current VR mode state - switch either into VR mode or exit from it, depending on the current state.

lookat(hlookat) / resetSensor(hlookat)
  • Reset the current sensor tracking to look to the given hlookat direction.
  • When using lookat() on changing panos to set the initial looking directing, the KEEPMOVING flag shouldn't be used.

calibrate(ondone*, onerror*)
  • Calibrate the gyroscope sensor manually (Mobile VR only).
  • Some devices have an uncalibrated or broken gyroscope sensor and this can lead to wrong unintended rotations (often called 'drifting').
  • When calling the calibrate() action the device should be absolutely still (!) for a few seconds (e.g. place the device on a flat and stable table for calibration). The orientation of the device itself doesn't matter for this process.
  • The mobilevr_autocalibration setting will get disabled when calling this action.
Parameters:
  • ondone (optionally)
    • Actions to call when the calibration is done.
  • onerror (optionally)
    • Actions to call when the calibration failed, e.g. when there was too much movement during the calibration.

resetCalibration()
  • Reset / zero the gyroscope sensor calibration data (Mobile VR only).

loadSettings(storage*)
 
saveSettings(storage*)
  • Load or save these WebVR / MobileVR settings:
  • The settings will be stored only locally in the browser on the current device.
  • A cross-domain HTML5 local storage will be used for this. As backup solution (if cross-domain access is not possible) the settings will be additionally also stored in the current domain local storage.
  • That means it would be only necessary to set the specific settings for a device / headset / user (IPD, lens, ...) only once - and all other krpano pages / sites will be automatically able to use the same settings.
  • When it was possible to load settings, the havesettings attribute will be set to true. But note - depending on from where the settings will be loaded and the speed of the network connection, there can be a delay until it will be set.
  • iOS Notes - the cross-domain local storage access is partially blocked in iOS Safari (depending on the privacy settings). There might be only a current-domain local storage available or no local storage at all in this case.
Parameter:
  • storage (optionally)
    • Set where to load or save the settings.
    • Available settings:
      • global (default) - load and save from a global cross-domain localstorage (and automatically also also the current domain localstorage as backup).
      • local - load and save only at the current domain localstorage. This can be used to avoid any external server access. 

update()
  • Update the current display settings.
  • This should be called when changing the display.stereo setting within the same frame.

WebVR Notes

'WebVR' is an experimental Javascript API that provides access to Virtual Reality devices in the browser. The WebVR API provides the head-tracking information and does the lens distortion for the VR headset.

Desktop Support (State: July 2017)
Today it is necessary to use special browsers builds for WebVR support. For the best experience at the moment the Firefox Nightly version for Windows would be recommended. This version provides currently the best performance and stability.

Android Support (State: July 2017)
For native WebVR support on Android the Chrome Canary browser is currently the best option.

GearVR Support
For GearVR support either use the normal Chrome browser and disable the 'GearVR Service' via the Package Disable Pro app to avoid that Oculus Home will be started when inserting the phone into the headset.
Or use the new GearVR 'Samsung Internet' browser and enable WebVR support there by entering internet://webvr-enable once.

MobileVR Notes

'MobileVR' is a krpano term and means using a normal smartphone for VR. The head tracking will be done using the accelerometer and gyroscope sensors and the rendering will be done via WebGL in the browser on the device.

Therefore the requirements of the mobile device and the mobile browser are:
  • Accelerometer + Gyroscope sensors,
  • devicemotion event support,
  • WebGL support,
  • and (optionally) HTML5 fullscreen mode support.
Additionally it would be good when the device has:
  • a good and large high-res screen
  • and lot of CPU and GPU power.

Android Chrome and Android Firefox are the recommended browsers for this. They provide the best support and performance.