Terrain Tiler fails to product tileset

Cesium ion
1

I’m trying to create a terrain tileset using v4.5.2 of the terrain-tiler.exe. Messages emitted by the tiler are not helpful in understanding the problem. The data is 1 meter lidar derived GeoTIFFs for Pennsylvania (275 GB zip files).

Command line:

C:\...\Software\Cesium\Cesium-ion-3D-Tiling-Pipeline-Windows-4.5.2\bin\terrain-tiler.exe --verbose --input-list H:\Tilesets\FileList.txt --height-reference MeanSeaLevel --output H:\Tilesets\PAMAP_LiDAR.terraindb > H:\Tilesets\PAMAP_verbose.txt 2>&1

Sampling of messages (duplicates removed for brevity):

e[0mIdentifying contiguous north-up raster sourcese[0m

e[0mStitching together contiguous north-up raster sourcese[0m

e[35mWarning: Warning 1: Too large for int: nane[0m
...
e[35mWarning: More than 1000 errors or warnings have been reported. No more will be reported from now.e[0m

e[0m13390 total raster sources.e[0m

e[33m0% complete.e[0m
e[0m(1 of 81) Adding 1.vrt to tileset. (Using 6352 sources: "38001640PAS_dem.tif", 

e[35mWarning: Raster contains invalid elevations that exceed max/min height defined on planet.e[0m

Sample dataset gdalinfo:

e[me[32me]9;8;"USERNAME"e\@e]9;8;"COMPUTERNAME"e\ e[92mJ:\PAMAP_LiDAR\NORTH\17002190PAN_deme[90m
e[90m>e[m e]9;12e\gdalinfo.exe 17002190PAN_dem.tif 
Driver: GTiff/GeoTIFF
Files: 17002190PAN_dem.tif
       17002190PAN_dem.aux
       17002190PAN_dem.rrd
       17002190PAN_dem.tif.aux.xml
Size is 3125, 3125
Coordinate System is:
PROJCRS["NAD83 / Pennsylvania North",
    BASEGEOGCRS["NAD83",
        DATUM["North American Datum 1983",
            ELLIPSOID["GRS 1980",6378137,298.257222101,
                LENGTHUNIT["metre",1]]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]],
        ID["EPSG",4269]],
    CONVERSION["unknown",
        METHOD["Lambert Conic Conformal (2SP)",
            ID["EPSG",9802]],
        PARAMETER["Latitude of false origin",40.1666666666667,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8821]],
        PARAMETER["Longitude of false origin",-77.75,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8822]],
        PARAMETER["Latitude of 1st standard parallel",41.95,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8823]],
        PARAMETER["Latitude of 2nd standard parallel",40.8833333333333,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8824]],
        PARAMETER["Easting at false origin",1968500,
            LENGTHUNIT["us_survey_feet",0.304800609601219],
            ID["EPSG",8826]],
        PARAMETER["Northing at false origin",0,
            LENGTHUNIT["us_survey_feet",0.304800609601219],
            ID["EPSG",8827]]],
    CS[Cartesian,2],
        AXIS["easting (X)",east,
            ORDER[1],
            LENGTHUNIT["us_survey_feet",0.304800609601219]],
        AXIS["northing (Y)",north,
            ORDER[2],
            LENGTHUNIT["us_survey_feet",0.304800609601219]]]
Data axis to CRS axis mapping: 1,2
Origin = (2190000.000000000000000,170000.000000000000000)
Pixel Size = (3.200000000000000,-3.200000000000000)
Metadata:
  AREA_OR_POINT=Area
  TIFFTAG_RESOLUTIONUNIT=1 (unitless)
  TIFFTAG_SOFTWARE=IMAGINE TIFF Support
Copyright 1991 - 1999 by ERDAS, Inc. All Rights Reserved
@(#)$RCSfile: etif.c $ $Revision: 1.10.1.9.1.9.2.11 $ $Date: 2004/09/15 18:42:01EDT $
  TIFFTAG_XRESOLUTION=1
  TIFFTAG_YRESOLUTION=1
Image Structure Metadata:
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  ( 2190000.000,  170000.000) ( 76d57' 7.31"W, 40d37'49.60"N)
Lower Left  ( 2190000.000,  160000.000) ( 76d57' 8.50"W, 40d36'10.80"N)
Upper Right ( 2200000.000,  170000.000) ( 76d54'57.63"W, 40d37'48.67"N)
Lower Right ( 2200000.000,  160000.000) ( 76d54'58.87"W, 40d36' 9.87"N)
Center      ( 2195000.000,  165000.000) ( 76d56' 3.08"W, 40d36'59.74"N)
Band 1 Block=64x64 Type=Float32, ColorInterp=Gray
  Min=382.690 Max=812.590 
  Minimum=382.690, Maximum=812.590, Mean=456.418, StdDev=89.927
  Overviews: 1563x1563, 782x782, 391x391, 196x196, 98x98, 49x49, 25x25
  Metadata:
    LAYER_TYPE=athematic
    STATISTICS_COVARIANCES=8086.79774158226
    STATISTICS_MAXIMUM=812.59002685547
    STATISTICS_MEAN=456.41761162347
    STATISTICS_MEDIAN=402.83999633789
    STATISTICS_MINIMUM=382.69000244141
    STATISTICS_MODE=384.79998779297
    STATISTICS_SKIPFACTORX=1
    STATISTICS_SKIPFACTORY=1
    STATISTICS_STDDEV=89.926624208753
2

Hello,

Thank you for providing those details. We have a theory as to what’s going wrong, but in order to confirm it we would need access to the source data to test internally. Would you be able to share it with us, through Google Drive or some other means?

Also, did the tiling job fail, or did it print those warnings but carry on?

Thanks,
Matt

3

It carried on and started to create a terraindb but those are all of the messages.

Sharing is difficult because the data is on an air-gapped network. Let me see if I can copy it to another drive so that it can be shared.

BTW, what’s your theory? Share privately if you like.

4

The data is available via FTP from ftp.pasda.psu.edu using anonymous login. You’ll need to copy the files from the directories in /pub/pasda/pamap/pamap_lidar/cycle1/DEM/{North,South}. They also provide mosaics for the counties which I’m preparing to tile today or tomorrow depending on how long it takes to unzip all of the files. I’m concerned with how the tiler will deal with overlapping datasets and possible NODATA values though.

5

Hello,

Thank you for sharing the data—download is in progress. We will update you once we have insight into the problem. Please let us know if you have additional details and/or more data to share in the meantime.

Thanks,
Matt

6

Recently started tiling a subset of 8 county DEMS that are neighbors, trying to avoid automatic logout this time.

7

Hi again,

Thanks for your patience—the download from FTP has been slow. It is, however, almost complete.

Is this the same data as asset ID 3957 in Cesium ion (PAMap Terrain), which is provided free to users? Do you need a local copy of this tiled data?

Thanks,
Matt

8

I can’t tell if it is the same (the link to PASDA isn’t helpful enough). If it is the 1-meter LiDAR I could use it, though going through the process of tiling such a large dataset has been informative.

BTW, the PDF that documents the tiler options is out of date in the bundle we get.

Lastly, can you provide a description of the decimation metric being used in the terrain tiler?

9

Sounds good, I’ll determine how we can make the data available to you and provide more info about it. I’m also seeing about the description of the decimation metric.

Thanks for the heads up about the PDF. Can you let me know the problem(s) you’ve noticed? Are options missing or incorrect?

13

Hi, below is some info about the data. The Original download links info at the bottom matches the FTP link you provided, so I believe this is the data you’re looking for. I’ll work on making the tiled version available to you.

PAMAP Program - 3.2 ft Digital Elevation Model

This dataset, produced by the PAMAP Program, consists of a raster digital elevation model with a horizontal ground resolution of 3.2 feet. The model was constructed from PAMAP LiDAR (Light Detection and Ranging) elevation points. PAMAP data are organized into blocks, which do not have gaps or overlaps, that represent 10,000 feet by 10,000 feet on the ground. The coordinate system for blocks in the northern half of the state is Pennsylvania State Plane North (datum:NAD83, units: feet); blocks in the southern half of the state are in Pennsylvania State Plane South. A block name is formed by concatenating the first four digits of the State Plane northing and easting defining the block’s northwest corner, the State identifier “PA”, and the State Plane zone designator “N” or “S” (e.g. 45001210PAS).
ion Asset ID 3957
Curator Pennsylvania Spatial Data Access
License The Originator, Publisher and Distributor exclude any and all implied warranties and make no warranty or representation with respect to the data files or accompanying documentation, including quality, performance, merchantability or fitness for a particular purpose. These data files and documentation are provided ‘as is’ and the User assumes the entire risk as to their quality and performance. The Originator and Publisher, as indicated in the metadata, should be clearly cited in any product derived from this data. Any modifications to this data must be described in any digital or hardcopy product derived from this data. link
Attribution PAMAP Program, Bureau of Topographic and Geologic Survey, PA Department of Conservation and Natural Resources
Attribution Source Pennsylvania Spatial Data Access | Full Metadata
Resolution 1m
Homepage http://www.pasda.psu.edu/uci/DataSummary.aspx?dataset=1247
Original download links ftp://ftp.pasda.psu.edu/pub/pasda/pamap/