iPhone Panorama with Forge Viewer

> Scan this QRCode with your iPhone camera to try now <

Follow @micbeale

Introduction:

I recently had a customer asking - 'how do I create an iPhone panorama with Forge Viewer'.  The idea is simple (see video)...

1. open iPhone camera
2. scan QR-Code
3. aim your iPhone up, down to look around
 

^{Bonus points - add 3D markers everywhere using the MarkupExt.js.}

How does it work ?

There's some quaternion math that ties the onDeviceOrientation events (in Euler degrees) from the iPhone compass/accelerometer, into the Forge Viewer's position/target camera object.

Anyone who's tried this, knows there's a bunch of ugly edge cases due to the portait/landscape modes and iPhone/Android differences.

Now, I know what you are thinking, "why don't you just use the webVR extension?"

True, but I need full-screen panorama, not the split-screen that webVR* offers. 

^{*And the webvr-polyfill, has been 'somewhat' unreliable between versions.}

Thankfully, the clever folks at Three.js have already figured out a robust solution already - deviceOrientation.js. 

I simply adapted this code to the Three.R71 camera/target model, and 'it just works'^TM

To minimize the code further, I re-use the 'gamepad' interface inside the 'first person' tool. I create a custom 'gamepad' component and override the camera-direction with the events from 'deviceOrientation' event. 

DEMO

How to use:

Steps:

1. Add <script src="deviceOrientationExt.js"></script> to your index.html
2. Activate 'first person' tool programmatically, like this...

viewer.toolController.activateTool('firstperson')

That's it.  

Generate a QR-Code:

Open your index.html page via your iPhone - or better yet, host your webpage publicly, then generate a QR-code and scan it with your iPhone/Android.

You can find an example of how to use the extension on GitHub here: https://github.com/wallabyway/deviceOrientationExt

Feel free to add any issues you find to my Github issues repo.

Follow @micbeale

Source Code:

(function() {
AutodeskNamespace('Autodesk.Viewing.Extensions.GamepadModule');

Autodesk.Viewing.Extensions.GamepadModule = function(viewer) {

    window.addEventListener('deviceorientation', e => this.deviceOrientation = e );

    this.update = function(camera) {
        if (!this.deviceOrientation) return camera;

        doe = this.deviceOrientation;
        euler.set(deg(doe.beta), deg(doe.alpha), -deg(doe.gamma), 'YXZ'); // 'ZXY' for the device, but 'YXZ' for us
        qe.setFromEuler(euler); // orient the device

        quaternion.set(Math.sqrt(0.5), 0, 0, Math.sqrt(0.5)); //  PI/2 around the x-axis
        quaternion.multiply(qe); // orient the device
        quaternion.multiply(q1); // camera looks out the back of the device, not the top
        quaternion.multiply(q0.setFromAxisAngle(zee, -this.orient)); // adjust for screen orientation
        m1.makeRotationFromQuaternion(quaternion);
        camera.setRotationFromMatrix(m1);

        // adjust camera target
        var lookAtDir = new THREE.Vector3(0, 0, -1);
        lookAtDir.applyQuaternion(camera.quaternion);
        camera.target = camera.position.clone().add(lookAtDir.clone().multiplyScalar(10));

        viewer.impl.invalidate(true, true, true);
        return camera;
    }

    const zee = new THREE.Vector3(0, 0, 1);
    const deg = THREE.Math.degToRad;
    let doe = null;
    let qe = new THREE.Quaternion();
    let q0 = new THREE.Quaternion();
    let q1 = new THREE.Quaternion(-Math.sqrt(0.5), 0, 0, Math.sqrt(0.5)); // - PI/2 around the x-axis
    let quaternion = new THREE.Quaternion();
    let euler = new THREE.Euler();
    var m1 = new THREE.Matrix4();
    this.orient = 0;
    window.addEventListener('orientationchange', e => this.orient = deg(window.orientation), false);

    this.activate = function(toolName) {};

    this.deactivate = function() {};
};
})();