This post is a manual for how to download and check the Copernicus DEM (CopDEM) 90 m version freely available from Amazon Web Services (AWS).
Introduction
The AWS version of CopDEM 90 m is downloaded outside Karttur’s GeoImagine Framework - all 26470 (!) tiles are downloaded with a single terminal command. To create a mosaic of these tiles, and then import (organise) them into the Framework is covered in this post.
Prerequisits
You must have installed Karttur’s GeoImagine Framework.
AWS download
Installing and running Amazon Web Services Command Line Interface 2 (AWSCLIV2) is covered in the post Global DEMs: access and download. Skipping the details here, all you have to do to download the CopDEM 90 m version from AWS is to open a Terminal window, change directory (cd) to where you want to save the downloaded CopDEM tiles, and run the command:
$ aws s3 cp s3://copernicus-dem-90m/ . --recursive --no-sign-request
A note on accessing the tiles
As noted above, the AWS dataset for CopDEM 90 consists of 26470 tif tiles. Each tile is located under an individual directory with the same name as the tif file, but without the .tif extension. The Finder app on my MacOS does not manage to display the content of the directory where I downloaded the AWS tile - there are simply too many. I can use other software with Graphical User Interface (GUI) to view the content, including MuCommander (available for all major operating systems). But in the end I turned to using terminal commands.
The download command above also retrieves some additional files apart from the tiles:
- readme.html
- tileList.txt (a list of all the tiles)
- grid.zip - a zip file with a vector shape file of all the tiles in geographic coordinates
To handle the files using the Terminal command line, cd to the parent folder where you downloaded the tiles (if you followed the steps above you are already in that very same directory already). To view readme.html
$ open readme.html
The tilelist is an ordinary text file (in this case similar to a csv file as there is only one column). To control the app to use when opening the text file, add the -a flag to the open command:
$ open -a atom tileList.txt
Unzip grid.zip with the command:
$ open grid.zip
This should explode the content of the zip file to a sub-directory calls grid. cd to the exploded data:
$ cd grid
and then open the shape file:
$ open dem90mGrid.shp
it turns out that the grid file actually do not contain the DEM tiles, it contains all the 1 degree tiles (260 * 180 = 64800) that cover the entire globe, including oceans.
Process chain
The process chain following the AWS download is available at Kartturs GitHub account and also found under the button below. The process chain is an ordinary text file linking to a series of json files. Empty rows and rows starting with a “#” are ignored. When called from karttur’s GeoImagine Framework, the processes of each json file are sequentially run. In the process chain below you can click on each json file name to see the complete content. The following sections also explain in more detail the processes of each json command file.
Bounding Boxes for raw tiles
The process BBoxTiledRawData creates bounding boxes for each raw tile. For the AWS CopDEM data that is not requires as the download also comes with a polygon vector with all the grids (see section on A note on accessing the tiles above). If, for some reason, you need to create vector polygons of the downloaded data, then you can use the process BBoxTiledRawData. Because the CopDEM AWS dataset contains so many tiles, the process takes quite a long time.
The process generates a single vector data source with all the included tiles, this time restricted to only the actual tiles that contain data.
As clearly seen in the figure above, the raw tiles are of arbitrary sizes with geographical coverage restricted to land. The next steps in the process chain converts these more arbitrary tiles to regular grids fitted to the projection system where you want to use the DEM.
A closer scrutiny of the north-western corner of the map reveals that the tiles do not cover the entire western hemisphere, but leave a gap west of -179 degrees longitude. The map to the left shows the part of Russia between -180 and -170 degrees longitude and how the most western parts are not covered by the tiles of the CopDEM 90 m version available from gisco-services.
Mosaic raw tiles
0125_MosaicAncillary_CopDEM-90m.json
Downloaded data that are delivered as arbitrary tiles need to be mosaicked together before importing to the framework. (SMAP, MODIS and Sentinel data do not need mosaicking of raw tiles as these datasets either are downloaded in predefined tiling systems or as complete global datasets). The mosaicking must be done as virtual (.vrt) dataset. To actually create a true global mosaic at e.g 90 m, as in this example, would take a long time and also take up unnecessary much disk space. The raw tile mosaic is only going to be used as an intermediate for creating the projection system predefined tiles that will later be used for all further analysis. The process MosaicAncillary does not import (organize in the terminology of the Framework) into the database, that is done in the next step.
Organise
0125_MosaicAncillary_CopDEM-30m.json
With the CopDEM 90 m data downloaded, unzipped and mosaicked, the data can be organised for (or imported to) the Framework. For the Copernicus DEM data, the process OrganizeAncillary as defined in the json object below only imports the virtual mosaic linked to the original (downloaded and unzipped) tiles.
Once you have run the json command, you can check in the Framework database, and you should find that the ancillary schema contains a layer with the compid dem_copdem.
Next step
The next step is tile the imported CopDEM to fit the projection system that you want to use for working with the data. The processing chain starting with the tiling, is, however, the same regardless of whihc version of CopDEM you choose to download, and is covered in the next instruction post.