Augmented Reality and Cesium

CesiumJS
#1

Hello everybody,

I want to use Cesium for an augmented reality project, which lets you see elements from the cesium simulation in AR.

Can anybody give me a hint on how to do this using the cesium API?

We have great VR capabilities in the viewer, but for AR I need to replace certain elements (sky & terrain) with a custom image (camera view).

I do not think this has been done before, at least my searches did not bring up anything related.

Thanks for your help!

#2

Hello,

If you look on our website, we have a lot of great tutorials and demos for getting started.

Take a look at these tutorials to get familiar with parts of Cesium: http://cesiumjs.org/tutorials.html

And then we have code examples for doing specific things here: http://cesiumjs.org/Cesium/Apps/Sandcastle/index.html?src=Hello%20World.html&label=Showcases

Let me know if you have any specific questions and I’ll be happy to help!

Best,

Hannah

#3

Hello Hannah, and thanks for your reply.
I know your tutorials as well es the Sandcastle examples, but none of these covers what I intend to achieve.

I need a way to embed a cesium scene in an existing live view, or at least access to the back buffer of the 3d scene.

Any hints are welcome.

#4

I just had an idea how to achieve this.
Could you give me your thoughts on wether this could work?

  • Display the webcam video (=background) in an HTML5-VideoElement

  • Set the video as texture for a 3D plane (as in the sandcastle video demo)

  • attach the plane to the viewers (virtual) camera, at a distance of ~500m scaled to fill the whole viewport

  • set terrain invisible (i.e. by setting a transparent texture)

might this work, or is it not performant enough for a smooth user experience?

#5

Did you get any where with this?

Cheers

#6

As a quick aside, it looks like it may also be possible to render Cesium on a transparent background with some modifications detailed here and Scene.backgroundColor.

#7

Interesting! We are currently using Unity for AR, but we are keen to start looking at Cesium…

#8

Awesome! We’d love to hear your use cases over in this thread so we can improve our support.

#9

A sandcastle example would be great to show this.

#10

While not full AR, we have a Sandcastle example for VR using Google Cardboard.

#11

Hi, here's a quick tutorial on how to create an AR app with Cesium:

#12

Curious to know if anyone has done anything with Cesium and AR since the last post.
The Spotup app mentioned seems to have disappeared from the app store.

I might have a need for such an app but I am guessing it is currently unsupported and so would require a lot of custom webgl coding correct.

I understand three.js can integrate with Cesium. Would this help given three.js has AR support

#13

I don’t think it would require much if any custom WebGL code. Setting of a split view for each eye is already possible with CesiumJS:

https://sandcastle.cesium.com/index.html?src=Cardboard.html

The part that needs to be integrating with some device and AR framework. Do you have any links to the ThreeJS AR support that you’ve been looking at? I believe ThreeJS integrates with other libraries to do this, as opposed to having to change much in the core engine, so I can imagine it’d be possible to make the same approach with CesiumJS.

#14

Did you manage to that?
Had similar idea with mobile phone camera for view utility, also trying to control camera (position, heading, pitch and rool) with mobile gyroscope and gps position (ok position with mobile phone is not precise but I have geodetic GNSS reciver to improve lat,lng and height.
Idea is like this picture:

test
test691×241 66.6 KB

#15

I got half way through but couldn’t complete it as it got complex. I am still interested in completing it. It’s not AR, it should be like the the globe should be locked in center and camera should move around it
Here is the demo video… I’ll provide the sandcastle or GitHub link soon

2 Likes
#16

This looks awesome! Can’t wat to see the sandcastle or Github link.

#17

You could achieve this with background video element. The only challenge here is to match the cesium field and angle of view with your camera. Maybe threejs ar could be a start as I think connecting the two libraries is supported. we have done something similar checkout this video https://youtu.be/XXmfeDzG1s0

#18

We’re controlling a Cesium camera with WebXR.
Here’s a demo.

1 Like
#19

Here you go!
https://github.com/mistatw/cesium-ar/blob/ebf5c755c1379c7be25f81b4e7db77d52003e79e/arcam.html

1 Like
#20

The github repo isn’t public.
Can’t see the content.