Introduction
In the Framework a system refers to a projection with predefined tiles that can be used for processing spatiotemporal datasets. This post first introduces the concept of a system and outlines the default systems, and then explains how to create a custom system.
Prerequisites
To run the tiling processes and produce the illustration in Fig. 1 you must have installed Karttur’s GeoImagine Framework.
Framework projection and tiling systems
Under the hood, almost all geospatial processing in Karttur’s GeoImagine Framework relates to a projection system and operates on tiles - regular grids of raster data. There is also a default system called system, for handling non-spatial data and arbitrary (usually Geographic) dataset.
To work with the Framework the user must have a pre-defined region (called tract) defined in a project and with the processing associated with a system. These parameters must be defined in the header of each json command file (see the post on Json elements, variables and objects for details):
{
"postgresdb": {
"db": "geoimagine"
},
"userproject": {
"userid": "karttur",
"projectid": "karttur",
"tractid": "karttur",
"siteid": "*",
"plotid": "*",
"system": "modis"
},
...
The tract can for instance be a continent, a country or a district within a country. To create a tract the user must associate it with a predefined default region. The predefined default regions are added when you setup the Framework, and include for instance all the world’s countries. When the Framework is set up, each default region is linked to the predefined tiles of the different projection systems used in the Framework. Thus for instance does the predefined region of the country Spain (es) link to 5 MODIS SIN grid tiles (Figure 1). Including the marine territory of Spain, additional two (2) tiles are linked. Spain is a country that spans two continents - Africa and Europe. The Framework contains separate default regions for the African and European parts of Spain, including both for the terrestrial and terrestrial+marine territories, also illustrated in figure 1.
Projection tiling system
Karttur’s GeoImagine Framework includes 2 default projection tiling systems: MODIS SIN GRID and MGRS UTM. In addition the Framework is also prepared for the three Equal-Area Scalable Earth (EASE) grid 2.0 systems: EASE-grid 2 North (ease2n), EASE-grid 2 South (ease2s) and EASE-grid 2 tropical (ease2t).
The MODIS, MGRS and EASE-grid 2 systems have different objectives. MODIS SIN grid projection system is for moderate to coarse resolution (100 m and coarser) global data, and the only consistent global projection. The three EASE-grid 2 projection systems together cover the globe, and are preferred over MODIS SIN Grid if the data is not truly global. Also the EASE-grid 2 systems are intended for coarse resolution data. MGRS is the system for fine scaled data, and has a much finer tiling system. MGRS can not be generated as a custom system, it is instead installed as a predefined system.
In addition to the above projection tiling systems, also the Landsat Worldwide Reference System (WRS) is included as a system, but rather as a scene location system, used only for retrieving and pre-processing Landsat satellite imagery.
EASE-grid 2 projection systems
db_setup includes definitions of tables directly related to EASE-grid 2 data, including data products acquired from the Soil Moisture Active Passive (SMAP) mission. db_setup also includes the complex setup of EASE-grid 2 tropical (with tiles of varying sizes to completely cover the Earth with no gaps or overlaps). The tiling systems for ease2n and ease2s, however, must be setup as custom projection systems. They will thus also serve as templates for setting up any other projection system in this manual. Compared to other custom systems you can not freely select the naming for the EASE-grid system, you must call them ease2n and ease2s.
Setup of tiling systems
Each tiling system is setup as a schema in the Framework postgres database. The core tables for defining the system as such (not its spatial data content) include the tables tilecoords (e.g. modis.tilecoords) and regions (e.g. modis.regions).
The table tilecoords defines all the tiles belonging to the projection system. In the projection system itself, the tiles are defined from 4 coordinates (minx, miny, maxx and maxy). Converted to geographic coordinates (longitude, latitude) the corners must be defined using separate longitude and latitude pairs.
MODIS
The MODIS tiling system contains 36 horizontal tile (originally called htile, but in the Framework changed to xtile) columns and 18 vertical tile (orignally vtile, but changed to ytile) rows. A large portion of these tiles, however, fall outside the Earthly domain. Including these dropouts can cause redundancy. The setup of the MODIS predefined tiling system is thus accomplished with a hardcoded script that is part of the setup_processes package.
The tile coordinates of the MODIS system are added to the database either by running the setup_processes.setup_processes_main.py with the alternative MODIS set to True (see the post Setup processes Part 2 Setup regions), or by using postgres database backup and restore.
For many applications with MODIS data, only tiles covering land (or continents excluding small islands) are of interest. To facilitate the use of the MODIS tiling system, the following default regions are added to the postgres database table modis.regions as part of db_setup (in the json parameter file modis_tile_regions_v090_sql.json):
- global
- land
- continent
These regions are required for adding further MODIS regions using the process LinkDefaultRegionTiles. The json parameter file regions-modtiles_v090.json loops over the default regions (including all versions of Spain above) and links the regions to the MODIS tiling. You can also add all the links between default regions and the MODIS tiles using the Framework process InsertTableCsv. If you set the parameters:
- "schema": "modis"
- "table": "regions"
the script will automatically search for the csv backup file
“DISKPATH”/dbdump/geoimagine/modis/regions/csv-geoimagine-modis-regions_YYYY-MM-DD.csv
and INSERT all the records found in that file.
DefineCustomSystem
With the process DefineCustomSystem you can define a new projection system including tiles. It will allow you to utilize the full potential of Karttur’s GeoImagine Framework for any region. In contrast to most other processes, DefineCustomSystem is only accessible from the package setup_processes.
A custom system is defined using the following parameters:
- systemid [a system identifier]
- epsg [the system epsg code]
- minx [the system minimum x coordinate]
- minx [the system minimum y coordinate]
- maxx [the system maximum x coordinate]
- maxy [the system maximum y coordinate]
- xtiles [nr of columns spanning from minimum to macximum x]
- ytiles [nr of rows spanning from minimum to macximum y]
Schemas and tables
Each projection system requires that the Framework postgres database have a schema with the same name (= systemid) as the system itself. The schema must then contains the following tables:
- tilecoords
- regions
- compdef
- compprod
- layer
- mask
The system table INSERT sql uses a predefined json command file that must be under the path ./systemdoc/system-setup_sql.json directly under the process_setup:
EASE-grid 2 north
As an example, the json command file for creating the EASE-grid 2 North (ease2n) custom system projection is available under the button.
To create a projection and tiling system also for EASE-grid 2.0 South, you only need to replace systemid to ease2s and epsg to 6932. Remember, EASE-grid 2.0 Tropical is preinstalled as the complex tiles require special solutions. Do not install ease2t using the Framework process DefineCustomSystem
SetupCustomGrids for EASE-grid 2
The json commands for setting up both EASE-grid North are prepared via the text file process_karttur_setup-ease-grid_YYYYMMDD.txt. Because the process includes several hudred reprojections it takes a bit longer to finish. You run it by just removing the comment sign for the call SetupCustomGrids(prodDB):
# To setup custom projection and tiling systems, remove the comment
#SetupCustomGrids(prodDB)
The function SetupCustomGrids links to process_karttur_setup-ease-grid_YYYYMMDD.txt. Inspect its content by toggling the button and click on the json file names.
If you want to setup your version of the Framework with support for EASE-grid 2.0, just remove the comment sing and rerun the module setup_process_main.py.
Custom projection tiling systems
Using the EASE-grid 2 projection tiling above as template, you can create any tiling system, as long as its projection has a proper epsg definition.
Next step
The next step is Database backup and restore.