Hi Alec,
The terrain elevations in Cesium are relative to the WGS84 ellipsoid. You can use the EGM96 geoid for a global MSL approximation to sample geoid heights relative to the WGS84 ellispoid, see here:
http://earth-info.nga.mil/GandG/wgs84/gravitymod/egm96/egm96.html
Alex