Vision Pro - Meshes render back to front

Cesium for Unity
1

Hi,

I’m looking at using Cesium on Vision Pro via Unity. Compiled native library and run it sucessfully simulator/device.

There is however an issue with rendering, it looks like the meshes on device are rendered back-to-front (see images).

Correct Unity Editor

image
image834×619 68.5 KB

Device - back to front (can see inside of buildings and terrain on the right has it’s ‘back’ rendered)

image
image975×719 95.8 KB

I suspect this will have to do with mesh winding order.

I’m trying to convert the indicies and found QuantizedMeshLoader::load, not 100% sure if that’s the best place to do so?

Or perhaps someone has better idea why is this happening and how it can be resolved?

Thanks

2

Found a way to change the order closer to Unity specific Mesh creation. Also as it’d seem that QuantizedMeshLoader is used for terrain.

If someone stumbles across this, there’s a UnityPrepareRendererResources.cpp in loadPrimitive method where Mesh data is set.

Changing the order on indicies array works.

3

Hi @ChrisHandzlik, welcome to the community!

Glad to hear you got it running on VisionPro! We’d welcome a pull request with any changes you needed to get that running.

I’ve heard of other people running into that reversed winding order problem, but I don’t totally understand why it happens. Cesium for Unity loads glTF models, which use a right-handed coordinate system. We transform the vertices in this right-handed coordinate system to Unity’s left-handed system via the game object’s Transform, which ends up having a negative scale. So presumably the VisionPro is considering the winding order differently from other parts of Unity when deciding between front and back faces. Perhaps it’s looking at winding order in world coordinates instead of model coordinates?

4

Thanks - let me see what I can do for PR, right now it’s not very automated. CMake was giving me quite a grief when trying to build for VisionOS so I build for IOS and changed target/sdk manually in XCode.

I think it only happenes when used in VisionPro in shared-space. When running windowed Unity app it’s fine.

From what I remember when using ‘windowed’ mode on vision pro Unity does rendering itself and their winding order is correct.

But when you run in shared space via Polyspatial, unity starts acting as a middle man as it can not render in that mode. RealityKit does all rendering and Polyspatial will try to convert meshes / shaders / etc to data structures that RealityKit understands. Likely somewhere there it doesn’t change order.

Might be not in-scope for cesium-for-unity as that part works, it’s only when Polyspatial is added and rendering happens in shared app space via RealityKit.

1 Like
5

Thanks for the background on windowed-via-Unity versus Polyspatial. That makes sense to me, and is really useful to know. I think both modes are in scope!

If it doesn’t feel polished enough for a PR, we’d also love to see some brief instructions describing the steps you went through to get it running for your own use. I think a lot of people would find that very useful. And perhaps it would be good to have those instructions yourself anyway for the future when you need to upgrade to a new version of the plugin? If you’re up for it, it could just be a thread on this forum.

1 Like
6

@Kevin_Ring Making some progress on automating that build with current workflow.

I’m getting stuck with
building for 'visionOS-simulator', but linking in object file (/Users/chris/.ezvcpkg/2024.07.12/installed/arm64-ios-unity/debug/lib/libs2.a[8](r2rect.cc.o)) built for 'iOS'

I’ve tried to change arm64-ios-unity.cmake to specify set(VCPKG_CMAKE_SYSTEM_NAME visionOS) and see if that’ll pull correct libraries but no luck. It fails with:

*** The output from the command was:
Computing installation plan...
The following packages will be built and installed:
    asyncplusplus:arm64-ios-unity@1.1#2 -- /Volumes/External/src/cesium-unity-samples-vp/Packages/com.cesium.unity/native~/vcpkg/ports/asyncplusplus
  * vcpkg-cmake:arm64-osx@2024-04-23
  * vcpkg-cmake-config:arm64-osx@2024-05-23
Additional packages (*) will be modified to complete this operation.
Detecting compiler hash for triplet arm64-osx...
Compiler found: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
Unable to determine toolchain use for arm64-ios-unity with with CMAKE_SYSTEM_NAME visionOS. Did you mean to use VCPKG_CHAINLOAD_TOOLCHAIN_FILE?
note: updating vcpkg by rerunning bootstrap-vcpkg may resolve this failure.
7

@ChrisHandzlik that might indicate you need to update CMake. CMake needs to know what CMAKE_SYSTEM_NAME=visionOS means, and that only happened in relatively recent versions.

8

Thanks @Kevin_Ring, here’s the draft PR that gets Cesium For Unity build support for Vision Pro, it has some workarounds though and is still work in progress.

Would be great if we could continue there to bring it to merge-able state.

1 Like
9

Thanks for opening it @ChrisHandzlik, much appreciated!