How to enable pan view again if I'm using camera.lookAtTransform

So i’m tinkering with a modified Cardboard example and it works great. My issue is that I want to snap out of the fixed camera and be able to drag and pan view.

I have searched that to snap out of camera.lookAt, simply call camera.lookAtTransform(Cesium.Matrix4.IDENTITY) and it you will be able to pan / drag again.

but doing this to cancel out camera.lookAtTransform, it will zoom you out of the globe and the camera is still locked.

should be similar to What should I do if the camera is locked when using lookAtTransform?