Warning if download expected to be large and/or when no bbox is provided

[jwass]

Should we warn the user if they don't provide a bbox or if the bbox is too large and we'll download a ton of data?

We'll have to look at when we can actually determine how much data will be downloaded

[gaeunfalo16]

@jwass
Hi! I'm trying to download whole world data by omitting bbox. After downloading, How much data will be loaded??
At this point, There is 1.1T in my server.

[mtravis]

I think from memory the total size will be 400GB for parquet so geojson will be 10x that which I assume that is the format you're attempting as it's over 1TB already.

What are your plans for the data once you have it all? That size of file is pretty unusable in geojson form.

[jwass]

@gaeunfalo16 If you want to download the whole world - you might just bypass using this library and download all the files directly. Using the aws cli you can just run aws --no-sign-request s3 sync s3://overturemaps-us-west-2/release/2024-06-13-beta.1/ . which will get it all.

You can find out how much data it is with:

$ aws --no-sign-request s3 ls --recursive --summarize s3://overturemaps-us-west-2/release/2024-06-13-beta.1/
...
Total Objects: 431
   Total Size: 454132596614

You can always subselect the themes/types you want if you don't want everything.

But as @mtravis pointed out - that's the Parquet data. If you want it all in GeoJSON it'll be much bigger and also much less useable IMO.

[gaeunfalo16]

@mtravis @jwass
Thank you for your comment! I tried to download in json format just because I may need it in case of future :)

• I think downloading json file is done. It takes 1.2T in my server

And download using aws works perfectly. Thank you!