Hi @stevediaz, welcome to the community!
In your input CityGML file, what are the elevations measured from? Are they heights above sea level, or heights above the WGS84 ellipsoid?
CesiumJS uses heights that are referenced to the WGS84 ellipsoid. See this previous thread on terrain elevations for more discussion.
If you can estimate the vertical error that you are seeing at a given longitude/latitude, we could get a quick check of whether the geoid vs ellipsoid error is enough to explain it.