Thanks Patrick! I do plan to do this for Cesium, for now without 3D buildings I could instead rise the camera many meters over the street and look at the imagery of the street side buildings with a 45deg tilt. All I have to do is a 45deg offset between the 2 view tilts. I’ll do a mockup video in GE for now to see how it works in practice.
One cool thing about Google Streetview is the ability to dynamically change FOV_zoom. You can’t dynamically change FOV_zoom in GE, but you can in Cesium. So once Cesium gets 3Dbuildings this splitscreen setup will be even better than it is now! (I say FOV_zoom since ‘zoom’ has become ambiguous: longitudinal translation or FOV camera zoom.)
I suppose using methods such as photogrammetry and LIDAR one is basically obtaining high resolution terrain data. One could view buildings simply as boxy mountains with very steep slopes! I suppose 3Dbuildings is a misnomer if that is their source. Everything is captured this way, be it trees, hills, vehicles, ships, bridges.