FEEDBACK REQUESTED: Native AI Copilot for Sandcastle

General
1

Hello Cesium Community,

The purpose of this post is to open a technical discussion regarding the next evolution for Sandcastle: integrating a native, agentic AI Copilot.

As AI-assisted development tools continue to evolve, we’re curious about your experiences using them with CesiumJS and Sandcastle. What has worked well? What challenges have you encountered?

The Vision: An Integrated AI Agent

We’re exploring whether a native AI agent integrated directly into Sandcastle could enhance the development experience. Our hypothesis is that tight integration with the Sandcastle environment could reduce iteration time and errors by enabling new capabilities, such as:

  • :building_construction: Rapid Scaffolding: Build applications with idiomatic, up-to-date CesiumJS code.
  • :lady_beetle: Intelligent Debugging: Read console output to suggest and apply fixes automatically.
  • :globe_with_meridians: Smart Asset Integration: Connect with Cesium ion to find and load appropriate assets for a given prompt.

Before diving into implementation, we want to ensure we’re building something that will have maximum impact for the community. Your ideas and feedback are essential.

:speech_balloon: We’d Love Your Feedback

Here are a few areas where your input would be especially helpful:

  1. Your Experience:
    Have you used any AI code generation tools (like GitHub Copilot, ChatGPT, etc.) while working with Sandcastle or CesiumJS? What worked well, and what didn’t?

  2. Ideal Use Cases:
    If you had an integrated Cesium AI agent, what is the first complex task you would ask it to build?

  3. Workflow Bottlenecks:
    What are the most repetitive or frustrating parts of your current Cesium development workflow that an agent could automate?

  4. Beyond Code Generation:
    What other tasks would you want an AI to handle? For example:

    • Managing camera views
    • Generating flight paths
    • Optimizing 3D Tiles styling
    • Something else?

  5. Vision & Concerns:
    What does the ideal AI-assisted development experience look like to you? What technical challenges or pitfalls should we avoid?

This is an open exploration, and your feedback will help shape this from a concept into a valuable tool for the entire community. Whether you’re excited, skeptical, or somewhere in between – we’d love to hear from you.

Looking forward to the discussion!

3 Likes
2

This is a great initiative. I am currently trying to integrate a CHAT GPT chat bot interface into my VR game so the user / gamer can speak to the AI BOT , ask questions about the area on the earth they are looking at etc… or ask to go to a particular spot on earth having descibed it. Currently I am using speech to tect to get a prompt for Geo Coding to return a longitude and latitude but more richness could be added through a chat gpt interface I am developing. This would require separate API licensing for GPT alongside that for Cesium ION so having everything integrated into Cesium with just one licence agreement for ION tile streaming and AI integration would be great if it offered the power of AI response offered through chat GPT :wink: Thanks

2 Likes