Control Cesium Camera with device orientation

1. A concise explanation of the problem you're experiencing.

Hey Guys,

I´m trying to control the position of the camera with the HTML5 device orientation (similar to https://groups.google.com/forum/#!topic/cesium-dev/cr2P2wfOwl4). The idea is that a user can move around his smartphone and
see for example terrain as if the camera would point out of the back of the device. I´ve searched around the web for quite a bit and found some useful sites (https://dev.opera.com/articles/w3c-device-orientation-usage/, http://www.users.on.net/~wallala/Cesium/Apps/Soarer/) but unfortunately I can not get it to work properly.

With the below code I can move the camera around by moving the device but the movements are not correct. Since I am stuck and not sure where exactly my problem is I thought I post this question and see if you guys can help me.

////////

var viewer = new Cesium.Viewer('cesiumContainer');
var scene = viewer.scene;
var camera = viewer.camera;

var terrainProvider = Cesium.createWorldTerrain();

function flyToLocation() {
  
  // Create callback for browser's geolocation
  function fly(position) {
    viewer.camera.flyTo({
      destination : Cesium.Cartesian3.fromDegrees(position.coords.longitude, position.coords.latitude, 10.0)
    });
  }

  // Ask browser for location, and fly there.
  navigator.geolocation.getCurrentPosition(fly);
};

var currentScreenOrientation = window.orientation || 0; // active default

if (window.DeviceOrientationEvent) {
  window.addEventListener('deviceorientation', onDeviceOrientationChanged, false);
}

/**
* Rotation Matrix functions
* Convert Yaw/alpha/Z, Pitch/beta/X, Roll/gamma/Y to and from rotation matrix, apply transformations

1 Like

Hey there,

Since you already have the rotation matrix, I don’t think the interim steps converting to heading, pitch, and roll are necessary.

Instead you can use Camera.setView with the endTransform option instead of orientation. You can construct the Matrix4 with Matrix4.fromRotationTranslation since you have both the translation (position) and the rotation matrix.

Otherwise to troubleshoot, make sure the matrix you are creating is passing the elements in the correct order (Cesium expects column-major order).

Sounds like an interesting project! Let us know if you have a demo once you get things working.

Thanks,

Gabby

Hey Gabby,

thanks for your help. I followed your advice and tried to set the view with the endTransform option. Unfortunately now the camera won´t move at all with the change of the device orientation. Again not sure what I am doing wrong since there are no error messages. Heres a code snippet:

function onDeviceOrientationChanged(eventData) {
  var beta = eventData.beta;
  var gamma = eventData.gamma;
  var alpha = eventData.alpha;
  
  var matrix = computeMatrix(alpha, beta, gamma, screenOrientation)
  var rotationmatrix = Cesium.Matrix3.fromArray(matrix);
  
  var matrix4 = new Cesium.Matrix4.fromRotationTranslation(rotationmatrix, destination, matrix4);
  
  console.log(matrix4);
  
  viewer.camera.setView({
    endTransform: matrix4
  });

};

Thanks again. Sorry if I´m doing some beginner´s mistake but im pretty new to Cesium.

One mistake I corrected was the order of the matrix you already mentioned. The code from Rich Tibbett creates the rotation matrix array in a row-major order. I fixed it by using

var rotationmatrix = new Cesium.Matrix3.fromRowMajorArray(matrix);

instead of

var rotationmatrix = Cesium.Matrix3.fromArray(matrix);

but this does not change the problem of the non-moving camera.

I experienced some strange behaviour. When I load the site the camera flies as expected to my location. The camera controls with the mouse work as expected with pan, rotate and zoom. If I then simulate a movement of the device via the Chrome developer tools sensor the camera controls don´t work the same way anymore. I cannot drag and by clicking and moving the mouse I rotate the camera. If I rotated the camera before I simulated movement the camera even changes position slightly.

I hope this does not sound confusing. If anyone has some ideas of what I am doing what or what I could try I would really appreciate it. Thank you.

Heres a Sandcastle-Link. Funnily enough the behaviour here es also a little bit different but I guess that because of the Sandcastle itself.

If anyone is interested I´ve managed to get it working for the most part. Heres the link to the gist: https://gist.github.com/AlexZeller/d9627fa25901580d1cc02dcefacfccd4

You can copy the code into an empty Sandcastle and it should work. If you view it on a smartphone you then should be able the move around the device and in theory the camera should move as if it would point out of the back.

Ih haven´t tested if the orientation is correct, and if the terrain that appears on the screen really is behind the phone. And theres still a problem if your phone rotates the screen.

Hi,

For the camera controls not working after you set the matrix, what I usually suggest is the following line to restore camera controls.

viewer.camera.lookAtTransform(Cesium.Matrix4.IDENTITY);

``

However, I think that will override your device orientation. Any reason why you still need the mouse controls if you are using the device to control orientation?

You can also override the default camera controls with your own camera controls.

For the slightly different behavior in sandcastle, are you using the latest version of Cesium for local development? Otherwise, it may have to do with the way the sandcastle app is set up, with panels for the live coding and a frame to run cesium itself.

Thanks! This looks like a cool use of device orientation and can have a lot of AR applications!

Gabby