Does Cesium for Omniverse v0.21 Support GLTF Point Clouds?

Cesium for Omniverse
1

I am trying to load a 3D Tileset which is composed of GLTF point clouds into the omniverse viewer via the Cesium for Omniverse extension without much success. The tilesets DO load properly when I try to view them through the CesiumJS viewer, however, they do not load when I try to read them locally through omniverse.

Unfortunately, I can’t release the data since it’s private, however, I did try to get some of the 3d Tile examples that contain points to load too without much success.

When I try to load this example from the github, it doesn’t work: 3d-tiles-samples/1.1/MultipleContents at main · CesiumGS/3d-tiles-samples · GitHub

I added an ECEF transform centered over the NVIDIA Building to put it into a geodetic coordinate frame. (This is also how I’m exporting my own data).

{
  "asset": {
    "version": "1.1"
  },
  "geometricError": 2.0,
  "root": {
    "contents": [
      {
        "uri": "planePoints.glb"
      }
    ],
    "boundingVolume": {
      "box": [
        0.5,
        -0.5,
        0.0,
        0.5,
        0.0,
        0.0,
        0.0,
        -0.5,
        0.0,
        0.0,
        0.0,
        0.1
      ]
    },
    "geometricError": 1.0,
    "transform": [
      1.8483488401529191,
      1.5294376690519829,
      -4.551115123125783e-17,
      1.0,
      1.3213675165273841,
      1.5149459055246444,
      1.7947034879151,
      1.0,
      1.4207459622292511,
      1.6741857822382546,
      1.6069978305526096,
      1.0,
      -2686890.0703628906,
      -4305361.19162711,
      3850350.3407306005,
      2.0
    ]
  }
}

When I use this dataset, I do see an error message pop up that I don’t see when I try with my own data.
2024-07-08 15:51:06 [Error] [omni.hydra] Skipping invalid mesh cesium_geometry_pool_2_object_92_context_1720449768. Reason: invalid points buffer, expected 0, got 1

In both cases, no data loads into the viewer.

2

Thanks for reaching out.

I tested with the example at https://github.com/CesiumGS/3d-tiles-samples/tree/main/glTF/EXT_structural_metadata/PropertyAttributesPointCloud. I was able to see the point cloud. I tested with Composer 2023.2.5-prod.3. What application and version of Kit are you working with?

3

I’m using USD composer 2023.2.5 with KIT 105.1.2 Release.

4

Actually, this is interesting…

If I point to the raw github url for the example you linked it works (haven’t tried adding the ecef transform yet). But if I download them to my local machine and try to point to the local file path, then I get an error.

The path I provide locally: file:///home/<user_name>/Downloads/example2/tileset.json

2024-07-10 18:36:57  [Error] [cesium.omniverse.plugin] [2024-07-10 14:36:57.320] [cesium-omniverse] [error] [TilesetContentManager.cpp:994] An unexpected error occurs when loading tile: curl: Couldn't read a file:// file

Update: The plot thickens…

When I tried pointing to the github url of the the previous Tileset 1.1 example, still nothing loads unlike the example you linked.

When I modify the example that you linked and provide an ECEF transform for the root node as I am trying to do with my own clouds, it half works… meaning, the house get’s transformed to roughly where I’d expect it (floating around the building), but the tree does not. The tree seems to have disappeared… (My only modification to the original is adding the ECEF transform at the end.)

{
  "asset": {
    "version": "1.1"
  },
  "schemaUri": "MetadataSchema.json",
  "statistics": {
    "classes": {
      "exampleMetadataClass": {
        "count": 29338,
        "properties": {
          "intensity": {
            "min": 0.0,
            "max": 0.6333333849906921,
            "mean": 0.28973701532415364,
            "median": 0.25416669249534607,
            "standardDeviation": 0.18222664489583626,
            "variance": 0.03320655011,
            "sum": 8500.30455558002,
            "_skewness": 0.4226942540824248,
            "_kurtosis": -0.230171075517038
          },
          "classification": {
            "occurrences": {
              "MediumVegetation": 6876,
              "Buildings": 22462
            }
          }
        }
      }
    }
  },  
  "geometricError" : 1024.0,
  "root" : {
    "boundingVolume" : {
      "box" : [ 7.422499656677246, 0.0, 3.9779999256134033, 7.422499656677246, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.9779999256134033 ]
    },
    "geometricError" : 512.0,
    "refine" : "ADD",
    "children" : [ {
      "boundingVolume" : {
        "box" : [ 4.793999671936035, 0.0, 3.9779999256134033, 4.793999671936035, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.9779999256134033 ]
      },
      "geometricError" : 0.0,
      "content" : {
        "uri" : "PropertyAttributesPointCloudHouse.gltf"
      }
    }, {
      "boundingVolume" : {
        "box" : [ 2.422499656677246, 0.0, 3.7230000495910645, 2.422499656677246, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.7230000495910645 ]
      },
      "geometricError" : 0.0,
      "transform" : [ 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 10.0, 0.0, 0.0, 1.0 ],
      "content" : {
        "uri" : "PropertyAttributesPointCloudTree.gltf"
      }
    } ],
    "transform": [
        0.8483488401529191,
        -0.5294376690519829,
        -5.551115123125783e-17,
        0.0,
        0.3213675165273841,
        0.5149459055246444,
        0.7947034879151,
        0.0,
        -0.4207459622292511,
        -0.6741857822382546,
        0.6069978305526096,
        0.0,
        -2686891.0703628906,
        -4305362.19162711,
        3850349.3407306005,
        1.0
    ]

  }
}
5

The previous Tileset 1.1 example uses pnts instead of glTF point clouds, which might explain the discrepancy. The house and tree differing is unexpected. Let me see if I can reproduce it…

6

Sorry just double checking… are GLTF pnts different from GLTF point clouds? The previous example still points to GLB files and the example describes them as GLTF files.

example

It’s not this pnts format which was deprecated in the 1.1 spec right? This is a 1.1 example and the file types look like GLTF files based on the .GLB suffix?

7

I just took a look, and I think I was mistaken. The sample isn’t pnts. It’s a glTF point cloud:

$ xxd planePoints.glb | head
00000000: 676c 5446 0200 0000 8413 0400 5803 0000  glTF........X...

The error for the file URI looks like a path error. What I’ve found can happen is that Omniverse attempt to look up the file path before it’s complete if the textbox looses focus. It’ll then have that error in the console. Can you try clearing the console and then re-focusing on the text field? I’ve also noticed on copy-paste that something the file URI is duplicated. It’s just not visible because the second entry is cropped by the UI. That might be the issue since I see the “file” token twice.

1 Like
8

I can confirm I’m not seeing anything loading with the Tileset 1.1 example. I get no error, though.

When I use your tileset.jon with the ECEF transform, it’s working for me. We’re synced on Composer and Kit versions. Can you try again to confirm? I wonder if it’s possibly a rendering artifact. You’re using the RTX - RealTime renderer?

Getting back to your actual data, can you confirm you get the Reason: invalid points buffer, expected 0, got 1 error on it as well as the Tileset 1.1 example? That’s an error I’m not able to reproduce.

9

image
image1920×667 49.3 KB

Yeah here’s an image of what I’m seeing. The house is floating above the building but the tree is no where to be found. It does look like the RTX real time rendering is whats being used.

For my actual data, the Reason: invalid points buffer, expected 0, got 1 error doesn’t appear. It actually doesn’t seem to be appearing anymore when I attempt to load either locally or from an online location. I’m not sure why that error’s gone away…

I still cannot load the Point Attributes Point Cloud you’ve linked from the local file system either. For some reason, every single time I type a letter, it seems like the error message is sent again. I tried deleting it all and retyping it entirely to avoid the double (hidden) paste you’ve described and still no luck.

Depending on what I put in, sometimes I see
2024-07-11 05:50:00 [Error] [cesium.omniverse.plugin] [2024-07-11 01:50:00.389] [cesium-omniverse] [error] [TilesetContentManager.cpp:994] An unexpected error occurs when loading tile: curl: Couldn't read a file:// file

and sometimes part way through typing I see
2024-07-11 05:49:55 [Error] [cesium.omniverse.plugin] [2024-07-11 01:49:55.771] [cesium-omniverse] [error] [TilesetContentManager.cpp:756] An unexpected error occurred when loading tile: curl: URL using bad/illegal format or missing URL

I did actually see

2024-07-11 05:52:47  [Error] [cesium.omniverse.plugin] [2024-07-11 01:52:47.893] [cesium-omniverse] [error] [ErrorList.h:68] Errors when loading tileset:
2024-07-11 05:52:47  [Error] [cesium.omniverse.plugin] - Error when parsing tileset JSON, error code 1 at byte offset 0

if I remove I provide the local directory hosted that contains the tileset.json file but not actually provide the tileset.json file itself. For example, providing /home/<user>/path/to/tileset/dir instead of /home/<user>/path/to/tileset/dir/tileset.json. This does seem like a separate (and reasonable) error…

Digging a little bit deeper it sort of does seem like the file:// error I’m seeing is related to this change log note from the cesium-native repo…

But this does seem to be from an old change… cesium-native v0.9.0 - 2021-11-01

10

Overall, the odd part to me is that (my) data renders fine when I use the CesiumJS sandbox viewers so I know it “works”.

It just doesn’t seem to be working with the Cesium for Omniverse extensions. At first I thought it was something specific to my data, but it looks like the results are inconsistent with the very simple 3D tile examples in the 3D tile sample repos too?

11

It’s been a while since I typed in a file:// URI, but while reading this I recalled that after each keystroke the event fires to load the file given the current input string. We’ve been looking at better UI events to hook into, but for the time being that’s why you’re seeing the error on each key input.

Is it possible for you to send over the USD file for your scene with the missing tree? It might be something particular to other data in the scene. I’ll see if I can reproduce the issue.

It just doesn’t seem to be working with the Cesium for Omniverse extensions. At first I thought it was something specific to my data, but it looks like the results are inconsistent with the very simple 3D tile examples in the 3D tile sample repos too?

We’ve also noticed some inconsistencies. At present, we render point clouds differently in Omniverse than CesiumJS. We’ve been syncing with NVIDIA to bring the techniques closer to parity. The upcoming KIT 106 release should help enable that. However, at present you should still be able to view glTF point clouds in a voxel-like technique. Let’s see if we can reproduce the tree issue and go from there.

12

Is it possible for you to send over the USD file for your scene with the missing tree? It might be something particular to other data in the scene. I’ll see if I can reproduce the issue.

I don’t think there’s anything specific about my USD file. I open a blank canvas and only do 2 things.

  1. Load the 3D Tile file from the URL like we’ve been doing.
  2. Load the Cesium OSM Buildings using a default token.

Loading the cesium OSM buildings is mainly to give me some geodetically pinned landmark to compare ensure that my ECEF transform is being applied in the way I expect.

We’ve also noticed some inconsistencies. At present, we render point clouds differently in Omniverse than CesiumJS. We’ve been syncing with NVIDIA to bring the techniques closer to parity

Ah got it. So this is somewhat “expected” (in the sense you’re aware that there are some problematic examples that are not fully covered)?

13

Oddly, the tree appears for me:

image
image1170×549 52.9 KB

What we’ve found is that some particular selections under the Render Settings tab can impact visibility for some geometry. I’ve never seen something partially visible, but that could be it. Would it be possible to send the scene’s .usda file so we can determine if some other setting is causing the issue?

Ah got it. So this is somewhat “expected” (in the sense you’re aware that there are some problematic examples that are not fully covered)?

I think that’s a fair statement. Some of these cases are being worked on community-wide. Rendering in Omniverse and the USD technology it’s built on works differently than what’s used on the Web with CesiumJS, or even Unreal and Unity. Fundamentally, Omniverse is designed to be a simulation engine as opposed to a game engine. To that end, rendering is only done via raytracing. The techniques to render points clouds in CesiumJS, Unreal, and Unity use raster-based techniques. Those shader-based techniques aren’t applicable to Omniverse’s raytraced approach. We’re developing analogous but differing techniques for point clouds. That’s why points are currently rendered as cubes.

14

Any update on whether the new the updates for Omniverse Point Clouds has been pushed out?