One high-level hint could be to consider glTF assets with the EXT_mesh_gpu_instancing extension. The idea is that you have “a few” tree models in one file, and just render them thousands of times, with different scales and orientations.
Creating such glTF assets from the raw data that you described may require some manual implementation work. But that approach could have the benefits of high performance (because GPU instancing is fast), and low memory overhead (because you only need few models, and just render them many times).
Another forum user applied that to trees in a city, in the thread around From I3dm to EXT_mesh_gpu_instancing? - #8 by bertt