Do you have “Enable World Bounds Checks” enabled in your project settings? If so, you will have to disable this, otherwise the game will try to pull you back into a limited “bounds” if you fly too far away.
If the issues persist even without this setting, is it possible to share a small example / video demonstration of the behavior, so we can try to reproduce it?