Is there a way to use the digital globe from my own server for free?

General
1

Hi Community.

I am working in a non-profit startup.

We basically want to provide a digital globe, parallel in a web app and later in a PC app.

The idea was to install a tile server on our webserver, download OSM Planet and OSM Buildings (if that’s not included in the planet download) and serve the tiles to our web app and our native PC app from our server (for cheap 0,005 cents a GB)

In the past weeks, I read a lot of Cesium articles and blog posts now, but I don’t really grasp, if our desire is possible and allowed???

If yes, how?

Could someone provide me with a set of the right links for what I need?

A combination of Cesium, OSM and Openlayers could be a solution maybe?
I read, serving tiles needs a lot of resources serverside (32-60GB of RAM), so streaming it from a free provider would be the best solution, if I didn’t misunderstand the concept of Openlayers???

Is a Contabo server a reliable and cheap solution for a tile server?

I know I am mixing up a lot here, but nothing is decided yet and I am open to every solution, as long as it is reliable and affordable!

2

FWIW, we had an internal app that needed to run completely offline. We generated our own tilemap files from some low-ish res public imagery sources, and our own terrain data tiles from publicly available terrain source data. I put the datasets into MBTiles files, and just wrote a small Python service to serve those tiles off disk.

I don’t know how much traffic you expect to be serving, but I can’t immediately see any reason why serving a bunch of tiles would require a massive amount of RAM at all. It should just be a glorified static file server - it’s just a question of how you’re storing those tile files on disk and reading those tile files when the request happens.

So, while I don’t know your actual business requirements, my assumption is you’re wayyy over-thinking this and definitely don’t need a hefty machine like that.

3

First, thank you very much for your reply.

We like to use a quality compared to the Cesium material with elevation, terrain and buildings. (see picture)

image
image1920×1080 804 KB

Elaborated from our advertising average click rate, we expect between 33.000-200.000 users a month using our service. (it varies a lot througout the year)

There is a studied Geo Engineer who has said, that a machine like this:

  • 8 CPU vCore
  • 32 GB RAM guaranteed
  • 800 GB SSD
  • 1 SSL-Zertifikat inklusive
  • Connection up to 500 MBit/s
  • Traffic Unlimited

…could serve a few hundred users. I guess that means something between 300 and 1000? :thinking:

But I am wondering too, because I can’t really think of having multiple hefty machines like this for 33.000 users??? :thinking: :thinking: :thinking:

Lets say 1 virtual machine = 1000 users…
…then i need 33 machines for 33.000 users…

Cost: 495€

Kind of cheap for 33 machines, but also expensive for just 33.000 users, because just a fraction of this users will spend money on my platform…

Maybe i miss the flexible scaling up and down part in my equation?
Like having a fixed stack and booking automatically more machines on high peaks…?

4

but short question: is there a tutorial available on how to replace the data source from cesium ion to my own server? same for the data source for the unreal plugin?

5

Can’t speak to the Unreal side at all For the Cesium.js toolkit, it’s “just” a matter of creating an instance of the right ImageryProvider and TerrainProvider with the URL of your own server and assigning them to the Viewer or Scene`.

If you’re wanting to host your own imagery and terrain (and potentially additional buildings and such), I would be surprised if you’d be able to match Cesium Ion’s quality - I know they’ve ingested a lot of very high-res imagery and terrain data for them to serve.

I think your best bet for server needs analysis would be to figure out how many tile requests get made by one typical user session, then start doing math from there. For example, some totally fake numbers I’m making up off the top of my head:

  • Initial load: 100 image tiles, 20 terrain tiles, in a span of 20 seconds, so an average of 6 reqs/sec
  • Peak users: 600 users in one hour
  • 10 new users per minute during peak
  • 10 users * 6 reqs/sec = 60 requests / sec

Like, those are totally fake numbers, but any web server application could serve 60 reqs/sec in its sleep, especially if it’s just a bunch of static files.

Now, to be clear: I’m definitely not an “experienced geo engineer”. I’ve worked on one internal project, so my own experience is relatively limited. So, take my advice with an appropriate grain of salt.

But I do think you’re over-estimating the needs of “200K users / month”. Narrow that down to “users at a time” and work out the math from there.

1 Like
6

Thanks a lot for your answer.

Yes you are right, I should rather go for ‘users at a time’.

And yes, collecting the high res material will be the next pain :sweat_smile:

7

that would rather mean that for 33.000 to 200.000 users a month, so average 1.100 to 6.666 users a day, i would rather end up with 2 - 10 machines, maybe running 2 constantly and spinning up up to 10 on peak times… means about 30€ to 150€ a month max.

that makes much more sense now.

Again, thank you very much for your input… :pray:

8

Hi,

We use offline server with elevation, maps and 3d tiles as you can see:

image
image1918×937 827 KB

9

Hi Ohad,

just saw your reply.

Is there documentation available to reproduce your tile server?

Best regards

10

Hi,

I can help you with the script that generate this tiles but first you need to look at this:

https://wiki.openstreetmap.org/wiki/Blender

If you have it then from there it is very simple to generated the tiles.

11

Cool. Thanks.

As I understand the documentation of blender-osm premium:
if I would choose for instance the area of Germany, Blender drops me the ready tiles?

12

blder has limitation on the size of one tile, but I can help you to create script that cut the area you wish to small tiles and genera5e you the cesium 3d tile pack

13

sounds awesome. thanks a lot.

Could you help me to build a pipeline even?

-Choose an OSM area in Blender
-Script generates cesium 3d tile pack
-serving the pack from my DigitalOcean server* to my Cesium JS Website**?

  • does it work with an s3 or does it have to be a droplet?
    ** Live Demo https://cloud-atlas.org (Google Login only, no states, but does its job)
14

Hi,

I can help you do create a basec pipleline based on the cesiumjs, but to create a massive one you need good architecture.

our solution is based on offline enviroment.

15

I am very happy about you spending time helping me.
I appreciate that a lot :slight_smile:

I use a Gaming Notebook for working in Unreal Engine, so local power is definitely there.
There is a server running on DigitalOcean already(2CPU,2GB RAM,2TB traffic), but I can switch easily to more power anytime. (20CPU, 48GB RAM, unlimited traffic 30€ a month, Strato V Server)

If I would understand how to run a tile server offline or a small one online, the classic way, that would help.
I need to understand the basics of how a tile server usually works.

Because, while working on the classic react template for the developer and entrepreneur community, I’m engineering a so-called dapp.

That is a decentralized app, where the back-end will run client-side, split on my app users devices and partly on the Ethereum blockchain.

Long story short: If you help me, understanding the pipeline, from choosing the map area, tiling it, to serving it the classic way, I will be able to transfer my knowledge to a p2p concept later.

You would basically help me, creating a free for everybody, decentralized, API tile server.

So everybody who wants to start a location-based service app or an app with a LBS feature, would get a free PWA-SPA-LOGIN-MAINSCREEN-MENU-Starter-Kit with Cesium integrated and will have access to a free global tile server, splitted on the users phones and the blockchain.

16

Do not wonder :sweat_smile:
After writing this to you, I changed the introduction text on my repository’s readme :joy:

17

Yes,

I will create reposetory with al the tools you need.

Ohad

18

Hi

look at this repo:

Ohad

19

Wow. I looked up the repository and I am so thankful that you support the project.
Its like a dream, you doing this :star_struck: :pray:

Best regards

Bo

1 Like
20

You can check out here aswell. Seems working