No, there are actually a lot more going on than simply a different projection matrix, even with shapes. For example, when geometry (for example polyline) crosses the antimeridian it needs to be specially broken up into separate parts for rendering, this is not a quick process and one of the reasons that setting the modelMatrix dynamically is not supported in 2D or CV mode. You can imagine that a model would have similar problems.
There’s also the actual visual appearance of the model itself. How should a large model look in Columbus View? Should it actually deform so that the model is a warped version of how it looks in real life, or should it look like it would in 3D mode? The warped version is accurate, but for many models wouldn’t look visually appealing or useful.
I don’t know too much about the actual model implementation, and I’m sure Patrick could provide additional details about what hurdles are involved, but Cesium in general does a lot more to support the different scene modes than simply change the projection.