ivert validate#

Validate one or more DEMs against ICESat-2 photon data downloaded to the local database.

For each input DEM, IVERT samples the nearest ICESat-2 photons within each grid cell, filters outliers, computes error statistics (bias, RMSE, NMAD, etc.), and writes plots and optionally GIS exports. Bathymetric and topographic DEMs are both supported.

Prerequisites#

You must have ICESat-2 data covering the DEM area already downloaded into the local database. If you haven’t done this yet, see ivert database.

Basic usage#

ivert validate mydem.tif

For a directory or glob pattern:

ivert validate /data/dems/survey_*.tif
ivert validate /data/dems/

Options#

Input / output#

Flag	Default	Description
`FILES_OR_DIRECTORY`	(required)	One or more GeoTIFF paths, a directory, or a glob pattern
`-o, --outdir DIR`	DEM’s own directory	Output directory for results files
`-bn, --band-num N`	`1`	Raster band to validate (1-indexed)
`--ndv VALUE`	(from file)	No-data value to exclude; use `nan` for NaN

Vertical datum#

Flag	Default	Description
`-V, --vdatum NAME`	(DEM’s embedded CRS)	Vertical datum of the DEM. Accepts EPSG code (`EPSG:5703`, `5703`), short name (`navd88`, `egm2008`, `mllw`, …), or transformez reference ID (`vdatum:mllw`)
`--list-vdatums`	—	Print all recognized vertical datum names and what each resolves to, then exit

ICESat-2 photons are stored either in WGS84 ‘ellipsoid’ (EPSG:4979) or EGM2008 ‘geoid’ (EPSG:3855) elevations, depending on the setting used (see ‘ivert options’ for more information). IVERT automatically transforms them to the DEM’s vertical datum before comparing. If the DEM’s CRS already encodes the vertical datum, -V is not needed.

The vertical datum shift is done by transformez. Tidal datums (mllw, mlw, msl, mhw, mhhw) are passed to it as its own NOAA VDatum references (vdatum:mllw, …) rather than as EPSG codes, and transformez decides how each surface is applied. IVERT checks that transformez can transform the datum you give before it starts, so an unsupported one stops the run straight away.

Photon filtering#

Flag	Default	Description
`-c, --classes CLASSES`	`1/6/40`	Slash-separated photon class codes to validate against (e.g. `1/40`). Photons in any other class are excluded before the elevation statistics are computed. See class codes, or run `ivert classes`
`-cl, --confidence-level N`	`4`	Minimum ATL03 signal confidence (1=low/keep all, 2=medium, 3=high, 4=very-high)
`-bc, --bathy-confidence F`	`0.90`	Minimum ATL24 bathymetry confidence for bathy-floor photons (0.0–1.0)
`-b, --buildings`	off	Include building-classed photons (class 7) in validation, on top of `-c/--classes`
`-sd, --outlier-sd F`	`2.5`	Outlier threshold in standard deviations (use `-1` to disable)
`-mp, --min-photons N`	`3`	Minimum photons a grid cell must contain to be validated. Cells with fewer are omitted from the results entirely

Bathymetry filters#

Before validating, IVERT removes class-40 (bathymetry floor) photons that ATL24 misclassified: surface returns over deep water, and noise below sea level on land. Four tests (deep, offshore, reference, land) check each photon against ETOPO 2022 and the OpenStreetMap landmask. All four are on by default; -bf none turns them off. Every threshold has its own --bathy-* flag and bathy_* setting. See Bathymetry filters for what each test does, how to tune it, and which files it downloads and keeps.

How per-cell statistics are computed#

Once a cell clears -mp/--min-photons, how its elevation is summarized depends on how many photons it holds:

Photons in cell	Treatment
1	The photon’s own height becomes the cell mean
2–4	The mean of all photons in the cell; too few to identify outliers
5 or more	Photons outside the interdecile (10th–90th percentile) range are dropped as outliers, and the mean is taken over the rest

The numphotons column always reports the raw photon count for the cell, while numphotons_intd reports how many were actually used in the statistics. The two are equal for cells that were not trimmed. Setting -mp 5 or higher guarantees every reported cell was interdecile-trimmed.

Output options#

Flag	Default	Description
`-ph, --include-photons`	off	Also write a point database of individual ICESat-2 photons used
`-ef, --export-formats FORMATS`	`tif,gpkg`	Comma-separated list of GIS error-export formats: `tif`, `gpkg`, `shp`, `xyz`. Use `none` or `""` to disable exports.

Coverage filtering#

On coarse DEMs (around 1 arc-second or coarser), one grid cell can hold photons from only a sliver of its area, so the cell’s ICESat-2 mean may not represent the whole cell. IVERT can measure each cell’s coverage: the fraction of a 15×15 grid of sub-cells that contain at least one photon. It can then drop cells whose coverage is too low.

Flag	Default	Description
`-mc, --measure-coverage`	off	Measure coverage per cell and add a `coverage_frac` column to the results
`-mcp, --minimum-coverage-pct P`	off	Drop cells with less than `P`% coverage (0–100), in every cell unless one of the flags below sets that cell type’s threshold
`-mcpl, --minimum-coverage-pct-land P`	off	Threshold for land cells only (cells with no bathymetry photons); overrides `-mcp` for them
`-mcpb, --minimum-coverage-pct-bathy P`	off	Threshold for bathymetry cells only (cells with any bathymetry photons); overrides `-mcp` for them

Any of the three thresholds turns on -mc by itself. The log reports how many cells of each type remained.

Bathymetry cells are few, and their photons are usually much sparser than land photons, so a threshold that suits land can remove nearly all of them. For example, -mcp 40 -mcpb 10 keeps land cells with at least 40% coverage and bathymetry cells with at least 10%. -mcpl 40 alone filters only land cells and keeps every bathymetry cell. Look at the coverage_frac column from a -mc run to choose thresholds: typical coverage depends strongly on the DEM’s cell size.

Labeling#

Flag	Default	Description
`-n, --name TEXT`	(DEM filename)	Region name shown on plots

Reproducing a run: manifests#

Flag	Default	Description
`-m, --manifest FILE`	—	Run with the settings recorded in a manifest from an earlier run

Every run that validates anything writes a manifest into its output directory: <dem>_ivert_manifest.ini for a single DEM, ivert_manifest.ini for a collection. It is a plain INI file you can read and edit. It records the IVERT version and the value of every option that can change the results, defaults included. Settings that fall back to your config are written out in full (the bathymetry filters and their thresholds, the export formats), so the manifest gives the same run on a machine with a different config. Options that don’t affect the results aren’t recorded (-o, -ow, --verbosity). The [run] section lists the command, input files and output directory for reference only. It is never applied.

To repeat a run, on the same DEMs or new ones, pass its manifest back:

ivert validate -m old_results/ivert_manifest.ini /data/new_dems/

Options given on the command line override the manifest’s values, so -m manifest.ini -sd 3 repeats a run with only the outlier threshold changed.

If the manifest’s options don’t match those of the installed IVERT (the manifest came from an older or newer version, or was edited), IVERT warns, names the version situation, and lists the options the manifest has that this version doesn’t and those it lacks. It then asks whether to go on without the unrecognized options, with current defaults for the missing ones. Answering no stops the run. When IVERT isn’t running in a terminal, it stops without asking.

A run that finds every DEM already validated (without -ow) does no work and leaves the existing manifest alone. When a collection run does work but its settings differ from the manifest already in the output directory, and some DEMs already have results, IVERT warns how many of them are reused rather than redone. Their results still reflect the old settings. Use -ow to redo them with the new ones.

Output files#

Per-DEM output files#

For a DEM named survey.tif, IVERT writes these files to the output directory:

File	Description
`survey_results.h5`	Full validation results (statistics, per-cell errors, photon matches)
`survey_results.png`	Multi-panel validation plot
`survey_summary_stats.txt`	Human-readable summary statistics (see below)
`survey_errors.tif`	GeoTIFF raster of mean per-cell error (if `tif` in export formats)
`survey_errors.gpkg`	GeoPackage point file of per-cell errors (if `gpkg` in export formats)
`survey_errors.shp`	Shapefile of per-cell errors (if `shp` in export formats)
`survey_errors.xyz`	Whitespace-delimited `x y error` text file (if `xyz` in export formats)
`survey_photons.gpkg`	Individual ICESat-2 photons used (if `-ph` flag given)
`survey_ivert_manifest.ini`	The run’s settings and IVERT version, for `-m/--manifest`

The _summary_stats.txt file contains:

Number of DEM cells validated
Total and mean number of ICESat-2 photons used
Mean bias error (DEM − ICESat-2) and RMSE
Full percentile breakdown of per-cell errors (0, 1, 5, 10, 20, 30, 40, 50, 60, 70, 80, 90, 95, 99, 100th percentiles)
Number of cells with bathymetry photons
Mean roughness (standard deviation of photon elevations within each cell)
The bathymetry filters applied, their thresholds, and how many class-40 photons each removed

Multi-DEM collection output files#

When validating multiple DEMs (a directory or glob pattern), IVERT also writes a set of collection-level summary files. These are named after the value passed to -n/--name; if no name is given, the prefix defaults to summary.

File	Description
`{name}_results.h5`	Combined HDF5 file of all per-cell results across every DEM in the collection
`{name}_individual_results.csv`	Per-DEM summary table: RMSE, mean bias, standard deviation, cell count, and mean photons per cell for each DEM
`{name}_plot.png`	Combined validation plot across all DEMs in the collection
`{name}_summary_stats.txt`	Same format as the per-DEM summary stats file, aggregated over all cells across all DEMs
`ivert_manifest.ini`	The run’s settings and IVERT version, for `-m/--manifest`. Per-DEM manifests are not written in a collection run

Examples#

Basic validation:

ivert validate mydem.tif

Specify vertical datum:

ivert validate mydem.tif -V navd88

Validate all DEMs in a directory, write results to a separate folder:

ivert validate /data/dems/ -o /data/results/

Export errors to all GIS formats:

ivert validate mydem.tif -ef tif,gpkg,shp,xyz

Disable error exports entirely:

ivert validate mydem.tif -ef none

Repeat an earlier run’s settings on new DEMs, with one option changed:

ivert validate -m /data/results/ivert_manifest.ini /data/more_dems/ -sd 3

List available vertical datum names:

ivert validate --list-vdatums