Processing compressed textures formats

[Tom_Lee]

Hello. All.

I’m doing testing with compressed textures formats in Cesium for Unity with cesium native

I’ve just tested my own 3D Tiles (b3dm) test datas which are ktx2 encoded and transcoded format (BC1(DXT1), ETC1) processing working fine in Cesium for Unity latest ( v1.2.0 )

I wonder if someone testing various compressed textures formats processing in code.

I guess that CesiumGS development team has a plan for this, becuase I read Cesium Omniverse this issue : Add support for compressed textures and mipmaps · Issue #266 · CesiumGS/cesium-omniverse · GitHub

I don’t do pull request about this because I don’t have experties about texture format and this thing needs battle test enough

any way I tested processing compressed textures formats with code below.

If anyone know better code than this just let me know!

diff --git a/native~/Runtime/src/TextureLoader.cpp b/native~/Runtime/src/TextureLoader.cpp
index c750be2..79a1ce5 100644
--- a/native~/Runtime/src/TextureLoader.cpp
+++ b/native~/Runtime/src/TextureLoader.cpp
@@ -20,12 +20,22 @@ namespace CesiumForUnityNative {

 UnityEngine::Texture
 TextureLoader::loadTexture(const CesiumGltf::ImageCesium& image) {
-  UnityEngine::Texture2D result(
-      image.width,
-      image.height,
-      UnityEngine::TextureFormat::RGBA32,
-      true,
-      false);
+  UnityEngine::TextureFormat textureFormat;
+
+  switch (image.compressedPixelFormat) {
+  case GpuCompressedPixelFormat::BC1_RGB:
+    textureFormat = UnityEngine::TextureFormat::DXT1;
+    break;
+  case GpuCompressedPixelFormat::ETC1_RGB:
+    textureFormat = UnityEngine::TextureFormat::ETC_RGB4;
+  default:
+    textureFormat = UnityEngine::TextureFormat::RGBA32;
+    break;
+  }
+
+  UnityEngine::Texture2D
+      result(image.width, image.height, textureFormat, false, false);
+
   result.hideFlags(UnityEngine::HideFlags::HideAndDontSave);

diff --git a/CesiumGltf/include/CesiumGltf/Ktx2TranscodeTargets.h b/CesiumGltf/include/CesiumGltf/Ktx2TranscodeTargets.h
index 4d0c7057..175f4a7c 100644
--- a/CesiumGltf/include/CesiumGltf/Ktx2TranscodeTargets.h
+++ b/CesiumGltf/include/CesiumGltf/Ktx2TranscodeTargets.h
@@ -78,7 +78,11 @@ struct CESIUMGLTF_API Ktx2TranscodeTargets {
    * @brief The gpu pixel compression format to transcode RGB ETC1S textures
    * into. If NONE, it will be decompressed into raw pixels.
    */
-  GpuCompressedPixelFormat ETC1S_RGB = GpuCompressedPixelFormat::NONE;
+#if defined(ANDROID_NDK)
+  GpuCompressedPixelFormat ETC1S_RGB = GpuCompressedPixelFormat::ETC1_RGB;
+#else
+  GpuCompressedPixelFormat ETC1S_RGB = GpuCompressedPixelFormat::BC1_RGB;
+#endif

   /**
    * @brief The gpu pixel compression format to transcode RGBA ETC1S textures
@@ -126,4 +130,4 @@ struct CESIUMGLTF_API Ktx2TranscodeTargets {
       bool preserveHighQuality);
 };

[Kevin_Ring]

You’re on the right track, except you shouldn’t actually modify cesium-native’s code. Instead, modify the values passed in by Cesium for Unity.

Here’s the equivalent code in Cesium for Unreal:

Telling cesium-native which formats are supported (it can vary depending on the particular hardware and RHI being used):

CesiumGS/cesium-unreal/blob/c83d3c1591c59004b3523e02a1ce52fb9bb2bfa6/Source/CesiumRuntime/Private/Cesium3DTileset.cpp#L1036

Identifying which format is being used when creating the Unreal texture:

CesiumGS/cesium-unreal/blob/c83d3c1591c59004b3523e02a1ce52fb9bb2bfa6/Source/CesiumRuntime/Private/CesiumTextureUtility.cpp#L545

I don’t know the details of how this would work in Unity (which is why isn’t not implemented yet), but it should be roughly similar.

Kevin

[Tom_Lee]

Thanks! @Kevin_Ring

I am going to check the way, passing values from Cesium for Unity to Cesium Natve

[Tom_Lee]

I changed code native~/Runtime/src/Cesium3DTilesetImpl.cpp instead of changing CesiumGltf/include/CesiumGltf/Ktx2TranscodeTargets.h

diff --git a/native~/Runtime/src/Cesium3DTilesetImpl.cpp b/native~/Runtime/src/Cesium3DTilesetImpl.cpp
index 720f982..2ae3fb2 100644
--- a/native~/Runtime/src/Cesium3DTilesetImpl.cpp
+++ b/native~/Runtime/src/Cesium3DTilesetImpl.cpp
@@ -412,6 +412,12 @@ void Cesium3DTilesetImpl::LoadTileset(
   TilesetContentOptions contentOptions{};
   contentOptions.generateMissingNormalsSmooth = tileset.generateSmoothNormals();

+#if defined(ANDROID_NDK)
+  contentOptions.ktx2TranscodeTargets.ETC1S_RGB = GpuCompressedPixelFormat::ETC1_RGB;
+#else
+  contentOptions.ktx2TranscodeTargets.ETC1S_RGB = CesiumGltf::GpuCompressedPixelFormat::BC1_RGB;
+#endif
+
   options.contentOptions = contentOptions;

   this->_lastUpdateResult = ViewUpdateResult();

I think I can use checking texture support code below ( csharp code side )

atteneder/KtxUnity/blob/f8b5db54f1aacce5372f2b7dbb4f362da4c1ccdb/Runtime/Scripts/TranscodeFormatHelper.cs#LL333C35-L333C35

I’ve not fully understood Reinterop yet.

so I need more time to passing values to c++ code (i.e. Cesium3DTilesetImpl.cpp )

[Tom_Lee]

@Kevin_Ring I wrote some code for checking gpu supporting texture format in C# code side.

and I ran this in command line in order to generate C# interop code

dotnet publish Reinterop~ -o .

nothing generated.

I already read your article and Readme.md this reinterop is quite difficult to me.

please tell me what to do more

C++ and C# Interop in Cesium for Unity

CesiumGS/cesium-unity/blob/main/Reinterop~/README.md

diff --git a/Runtime/Cesium3DTileset.cs b/Runtime/Cesium3DTileset.cs
index c9fd3f2..23ad97e 100644
--- a/Runtime/Cesium3DTileset.cs
+++ b/Runtime/Cesium3DTileset.cs
@@ -1,6 +1,7 @@
 using Reinterop;
 using System;
 using UnityEngine;
+using UnityEngine.Experimental.Rendering;

 namespace CesiumForUnity
 {
@@ -632,6 +633,167 @@ namespace CesiumForUnity
             }
         }

+        [SerializeField]
+        private bool _ETC1_RGB = false;
+        public bool ETC1_RGB
+        {
+            get => this._ETC1_RGB;
+            set
+            {
+                this._ETC1_RGB = value;
+            }
+        }
+        [SerializeField]
+        private bool _ETC2_RGBA = false;
+        public bool ETC2_RGBA
+        {
+            get => this._ETC2_RGBA;
+            set
+            {
+                this._ETC2_RGBA = value;
+            }
+        }
+        [SerializeField]
+        private bool _BC1_RGB = false;
+        public bool BC1_RGB
+        {
+            get => this._BC1_RGB;
+            set
+            {
+                this._BC1_RGB = value;
+            }
+        }
+        [SerializeField]
+        private bool _BC3_RGBA = false;
+        public bool BC3_RGBA
+        {
+            get => this._BC3_RGBA;
+            set
+            {
+                this._BC3_RGBA = value;
+            }
+        }
+        [SerializeField]
+        private bool _BC4_R = false;
+        public bool BC4_R
+        {
+            get => this._BC4_R;
+            set
+            {
+                this._BC4_R = value;
+            }
+        }
+        [SerializeField]
+        private bool _BC5_RG = false;
+        public bool BC5_RG
+        {
+            get => this._BC5_RG;
+            set
+            {
+                this._BC5_RG = value;
+            }
+        }
+        [SerializeField]
+        private bool _BC7_RGBA = false;
+        public bool BC7_RGBA
+        {
+            get => this._BC7_RGBA;
+            set
+            {
+                this._BC7_RGBA = value;
+            }
+        }
+        [SerializeField]
+        private bool _PVRTC1_4_RGB = false;
+        public bool PVRTC1_4_RGB
+        {
+            get => this._PVRTC1_4_RGB;
+            set
+            {
+                this._PVRTC1_4_RGB = value;
+            }
+        }
+        [SerializeField]
+        private bool _PVRTC1_4_RGBA = false;
+        public bool PVRTC1_4_RGBA
+        {
+            get => this._PVRTC1_4_RGBA;
+            set
+            {
+                this._PVRTC1_4_RGBA = value;
+            }
+        }
+        [SerializeField]
+        private bool _ASTC_4x4_RGBA = false;
+        public bool ASTC_4x4_RGBA
+        {
+            get => this._ASTC_4x4_RGBA;
+            set
+            {
+                this._ASTC_4x4_RGBA = value;
+            }
+        }
+        [SerializeField]
+        private bool _PVRTC2_4_RGB = false;
+        public bool PVRTC2_4_RGB
+        {
+            get => this._PVRTC2_4_RGB;
+            set
+            {
+                this._PVRTC2_4_RGB = value;
+            }
+        }
+        [SerializeField]
+        private bool _PVRTC2_4_RGBA = false;
+        public bool PVRTC2_4_RGBA
+        {
+            get => this._PVRTC2_4_RGBA;
+            set
+            {
+                this._PVRTC2_4_RGBA = value;
+            }
+        }
+        [SerializeField]
+        private bool _ETC2_EAC_R11 = false;
+        public bool ETC2_EAC_R11
+        {
+            get => this._ETC2_EAC_R11;
+            set
+            {
+                this._ETC2_EAC_R11 = value;
+            }
+        }
+        [SerializeField]
+        private bool _ETC2_EAC_RG11 = false;
+        public bool ETC2_EAC_RG11
+        {
+            get => this._ETC2_EAC_RG11;
+            set
+            {
+                this._ETC2_EAC_RG11 = value;
+            }
+        }
+
+        public void CheckSupportedGpuCompressedPixelFormats()
+        {
+            ETC1_RGB = SystemInfo.IsFormatSupported(GraphicsFormat.RGB_ETC_UNorm, FormatUsage.Sample);
+            ETC2_RGBA = SystemInfo.IsFormatSupported(GraphicsFormat.RGBA_ETC2_SRGB, FormatUsage.Sample);
+            BC1_RGB = SystemInfo.IsFormatSupported(GraphicsFormat.RGBA_DXT1_SRGB, FormatUsage.Sample);
+            BC3_RGBA = SystemInfo.IsFormatSupported(GraphicsFormat.RGBA_DXT5_SRGB, FormatUsage.Sample);
+            BC4_R = SystemInfo.IsFormatSupported(GraphicsFormat.R_BC4_SNorm, FormatUsage.Sample);
+            BC5_RG = SystemInfo.IsFormatSupported(GraphicsFormat.RG_BC5_SNorm, FormatUsage.Sample);
+            BC7_RGBA = SystemInfo.IsFormatSupported(GraphicsFormat.RGBA_BC7_SRGB, FormatUsage.Sample);
+            PVRTC1_4_RGB = SystemInfo.IsFormatSupported(GraphicsFormat.RGB_PVRTC_4Bpp_SRGB, FormatUsage.Sample);
+            PVRTC1_4_RGBA = SystemInfo.IsFormatSupported(GraphicsFormat.RGBA_PVRTC_4Bpp_SRGB, FormatUsage.Sample);
+            ASTC_4x4_RGBA = SystemInfo.IsFormatSupported(GraphicsFormat.RGBA_ASTC4X4_SRGB, FormatUsage.Sample);
+
+            //I don't know why these formats don't exist in UnityEngine.Experimental.Rendering.GraphicsFormat enum
+            //PVRTC2_4_RGB = false;
+            //PVRTC2_4_RGBA = false;
+            //ETC2_EAC_R11 = false;
+            //ETC2_EAC_RG11 = false;
+        }
+

if this thing work I put these code in c++ side and testing this on runtime

diff --git a/native~/Runtime/src/Cesium3DTilesetImpl.cpp b/native~/Runtime/src/Cesium3DTilesetImpl.cpp
index 720f982..381c02d 100644
--- a/native~/Runtime/src/Cesium3DTilesetImpl.cpp
+++ b/native~/Runtime/src/Cesium3DTilesetImpl.cpp
@@ -412,6 +412,30 @@ void Cesium3DTilesetImpl::LoadTileset(
   TilesetContentOptions contentOptions{};
   contentOptions.generateMissingNormalsSmooth = tileset.generateSmoothNormals();

+  tileset.CheckSupportedGpuCompressedPixelFormats();
+
+  CesiumGltf::SupportedGpuCompressedPixelFormats supportedFormats;
+  supportedFormats.ETC1_RGB = tileset.ETC1_RGB();
+  supportedFormats.ETC2_RGBA = tileset.ETC2_RGBA();
+  supportedFormats.BC1_RGB = tileset.BC1_RGB();
+  supportedFormats.BC3_RGBA = tileset.BC3_RGBA();
+  supportedFormats.BC4_R = tileset.BC4_R();
+  supportedFormats.BC5_RG = tileset.BC5_RG();
+  supportedFormats.BC7_RGBA = tileset.BC7_RGBA();
+  supportedFormats.ASTC_4x4_RGBA = tileset.ASTC_4x4_RGBA();
+  supportedFormats.PVRTC2_4_RGBA = tileset.PVRTC2_4_RGBA();
+  supportedFormats.ETC2_EAC_R11 = tileset.ETC2_EAC_R11();
+  supportedFormats.ETC2_EAC_RG11 = tileset.ETC2_EAC_RG11();
+
+  contentOptions.ktx2TranscodeTargets =
+      CesiumGltf::Ktx2TranscodeTargets(supportedFormats, false);
+

[Kevin_Ring]

Hmm I’m not sure where you’re lost.

First, you need to understand how to build Cesium for Unity. The developer instructions cover that:

CesiumGS/cesium-unity/blob/main/Documentation~/developer-setup.md

Now if you need to do interop with something new (for example, access a new C# class or method from C++ code), then you need to add that new thing to ConfigureReinterop.cs, as it explains in the docs I linked above.

Then you launch Unity, which runs the C# compiler, which generates the C# and C++ code necessary for this to work. Building Reinterop won’t generate code. Reinterop is a plugin for the C# compiler. The code doesn’t get generated until Reinterop runs inside the C# compiler, which runs inside Unity.

[Tom_Lee]

I succeeded call C# method from C++ side

I just missed adding code in ConfigureReinterop.cs, you mentioned.

I tested my code in runtime. I found problems. I need more time to fix this.

Thanks!

[Tom_Lee]

just sharing my progress.

I’m testing my code with compression format BC1, BC7 and ETC1

I coudn’t test with ASTC I don’t have specific b3dm data set (gltf which has UASTC encoded texture) ,

I coudn’t test with ETC2 I don’t have specific b3dm data set (gltf which has ETC1S encoded texture with alpha)

I coudn’t test with PVRTC1,2 (I don’t have these specific formats supported devices)

If somebody better code than mine or confident with all compression format just do pull request

[Tom_Lee]

I created a pull request. review please.

adding code for texture GpuCompressedPixelFormats