APEx STAC catalogue editor guide

This notebook demonstrates how to add metadata to an APEx STAC catalogue.

The STAC API is a standard, so other STAC documentation is also relevant.

Creating STAC metadata from scratch

In this first part, we show how to start from zero with a very basic example.

In most real-world cases, it is not needed nor recommended to author STAC metadata from scratch directly. Instead, various tools and platforms will generate STAC metadata for you. The STAC metadata generated here is entirely fictional, and also very much useless. It is not an example of how to create high quality metadata that complies with FAIR principles.

This example mostly serves as a demonstration of the API, which is very simply to use. Most of the more complex examples and tools will avoid this type of direct interaction.

import requests
from owslib.ogcapi.records import Records
from owslib.util import Authentication

class BearerAuth(requests.auth.AuthBase):
    def __init__(self, token):
        self.token = token
    def __call__(self, r):
        r.headers["authorization"] = "Bearer " + self.token
        return r
    


auth = BearerAuth("""PROVIDE_OIDC_TOKEN""")

r = Records("https://catalogue.project-a.apex.esa.int",auth=Authentication(auth_delegate=auth))
r

<owslib.ogcapi.records.Records at 0x7809e804b050>

To get some information about our daaset, we can run a simple gdalinfocommand against a representative asset.

!GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR gdalinfo -json "/vsicurl/https://eoresults.esa.int/d/APEX_TEST/2020/03/01/europe_aggr-orgc_00-020_mean_100_202003-202210/europe_aggr-orgc_00-020_mean_100_202003-202210.tif"

{
  "description":"/vsicurl/https://eoresults.esa.int/d/APEX_TEST/2020/03/01/europe_aggr-orgc_00-020_mean_100_202003-202210/europe_aggr-orgc_00-020_mean_100_202003-202210.tif",
  "driverShortName":"GTiff",
  "driverLongName":"GeoTIFF",
  "files":[
    "/vsicurl/https://eoresults.esa.int/d/APEX_TEST/2020/03/01/europe_aggr-orgc_00-020_mean_100_202003-202210/europe_aggr-orgc_00-020_mean_100_202003-202210.tif"
  ],
  "size":[
    40888,
    41712
  ],
  "coordinateSystem":{
    "wkt":"PROJCRS[\"ETRS89-extended / LAEA Europe\",\n    BASEGEOGCRS[\"ETRS89\",\n        ENSEMBLE[\"European Terrestrial Reference System 1989 ensemble\",\n            MEMBER[\"European Terrestrial Reference Frame 1989\"],\n            MEMBER[\"European Terrestrial Reference Frame 1990\"],\n            MEMBER[\"European Terrestrial Reference Frame 1991\"],\n            MEMBER[\"European Terrestrial Reference Frame 1992\"],\n            MEMBER[\"European Terrestrial Reference Frame 1993\"],\n            MEMBER[\"European Terrestrial Reference Frame 1994\"],\n            MEMBER[\"European Terrestrial Reference Frame 1996\"],\n            MEMBER[\"European Terrestrial Reference Frame 1997\"],\n            MEMBER[\"European Terrestrial Reference Frame 2000\"],\n            MEMBER[\"European Terrestrial Reference Frame 2005\"],\n            MEMBER[\"European Terrestrial Reference Frame 2014\"],\n            MEMBER[\"European Terrestrial Reference Frame 2020\"],\n            ELLIPSOID[\"GRS 1980\",6378137,298.257222101,\n                LENGTHUNIT[\"metre\",1]],\n            ENSEMBLEACCURACY[0.1]],\n        PRIMEM[\"Greenwich\",0,\n            ANGLEUNIT[\"degree\",0.0174532925199433]],\n        ID[\"EPSG\",4258]],\n    CONVERSION[\"Europe Equal Area 2001\",\n        METHOD[\"Lambert Azimuthal Equal Area\",\n            ID[\"EPSG\",9820]],\n        PARAMETER[\"Latitude of natural origin\",52,\n            ANGLEUNIT[\"degree\",0.0174532925199433],\n            ID[\"EPSG\",8801]],\n        PARAMETER[\"Longitude of natural origin\",10,\n            ANGLEUNIT[\"degree\",0.0174532925199433],\n            ID[\"EPSG\",8802]],\n        PARAMETER[\"False easting\",4321000,\n            LENGTHUNIT[\"metre\",1],\n            ID[\"EPSG\",8806]],\n        PARAMETER[\"False northing\",3210000,\n            LENGTHUNIT[\"metre\",1],\n            ID[\"EPSG\",8807]]],\n    CS[Cartesian,2],\n        AXIS[\"northing (Y)\",north,\n            ORDER[1],\n            LENGTHUNIT[\"metre\",1]],\n        AXIS[\"easting (X)\",east,\n            ORDER[2],\n            LENGTHUNIT[\"metre\",1]],\n    USAGE[\n        SCOPE[\"Statistical analysis.\"],\n        AREA[\"Europe - European Union (EU) countries and candidates. Europe - onshore and offshore: Albania; Andorra; Austria; Belgium; Bosnia and Herzegovina; Bulgaria; Croatia; Cyprus; Czechia; Denmark; Estonia; Faroe Islands; Finland; France; Germany; Gibraltar; Greece; Hungary; Iceland; Ireland; Italy; Kosovo; Latvia; Liechtenstein; Lithuania; Luxembourg; Malta; Monaco; Montenegro; Netherlands; North Macedonia; Norway including Svalbard and Jan Mayen; Poland; Portugal including Madeira and Azores; Romania; San Marino; Serbia; Slovakia; Slovenia; Spain including Canary Islands; Sweden; Switzerland; Türkiye (Turkey); United Kingdom (UK) including Channel Islands and Isle of Man; Vatican City State.\"],\n        BBOX[24.6,-35.58,84.73,44.83]],\n    ID[\"EPSG\",3035]]",
    "dataAxisToSRSAxisMapping":[
      2,
      1
    ]
  },
  "geoTransform":[
    2515000.0,
    100.0,
    0.0,
    5512800.0,
    0.0,
    -100.0
  ],
  "metadata":{
    "":{
      "AREA_OR_POINT":"Area"
    },
    "IMAGE_STRUCTURE":{
      "LAYOUT":"COG",
      "COMPRESSION":"DEFLATE",
      "INTERLEAVE":"BAND",
      "PREDICTOR":"2"
    }
  },
  "cornerCoordinates":{
    "upperLeft":[
      2515000.0,
      5512800.0
    ],
    "lowerLeft":[
      2515000.0,
      1341600.0
    ],
    "lowerRight":[
      6603800.0,
      1341600.0
    ],
    "upperRight":[
      6603800.0,
      5512800.0
    ],
    "center":[
      4559400.0,
      3427200.0
    ]
  },
  "wgs84Extent":{
    "type":"Polygon",
    "coordinates":[
      [
        [
          -35.0421748,
          67.1549879
        ],
        [
          -9.3033749,
          33.067326
        ],
        [
          34.1876844,
          31.8603398
        ],
        [
          62.8622654,
          64.3475852
        ],
        [
          -35.0421748,
          67.1549879
        ]
      ]
    ]
  },
  "bands":[
    {
      "band":1,
      "block":[
        512,
        512
      ],
      "type":"UInt16",
      "colorInterpretation":"Gray",
      "noDataValue":65535.0,
      "overviews":[
        {
          "size":[
            20444,
            20856
          ]
        },
        {
          "size":[
            10222,
            10428
          ]
        },
        {
          "size":[
            5111,
            5214
          ]
        },
        {
          "size":[
            2555,
            2607
          ]
        },
        {
          "size":[
            1277,
            1303
          ]
        },
        {
          "size":[
            638,
            651
          ]
        },
        {
          "size":[
            319,
            325
          ]
        }
      ],
      "metadata":{}
    }
  ],
  "stac":{
    "proj:shape":[
      41712,
      40888
    ],
    "proj:wkt2":"PROJCRS[\"ETRS89-extended / LAEA Europe\",\n    BASEGEOGCRS[\"ETRS89\",\n        ENSEMBLE[\"European Terrestrial Reference System 1989 ensemble\",\n            MEMBER[\"European Terrestrial Reference Frame 1989\"],\n            MEMBER[\"European Terrestrial Reference Frame 1990\"],\n            MEMBER[\"European Terrestrial Reference Frame 1991\"],\n            MEMBER[\"European Terrestrial Reference Frame 1992\"],\n            MEMBER[\"European Terrestrial Reference Frame 1993\"],\n            MEMBER[\"European Terrestrial Reference Frame 1994\"],\n            MEMBER[\"European Terrestrial Reference Frame 1996\"],\n            MEMBER[\"European Terrestrial Reference Frame 1997\"],\n            MEMBER[\"European Terrestrial Reference Frame 2000\"],\n            MEMBER[\"European Terrestrial Reference Frame 2005\"],\n            MEMBER[\"European Terrestrial Reference Frame 2014\"],\n            MEMBER[\"European Terrestrial Reference Frame 2020\"],\n            ELLIPSOID[\"GRS 1980\",6378137,298.257222101,\n                LENGTHUNIT[\"metre\",1]],\n            ENSEMBLEACCURACY[0.1]],\n        PRIMEM[\"Greenwich\",0,\n            ANGLEUNIT[\"degree\",0.0174532925199433]],\n        ID[\"EPSG\",4258]],\n    CONVERSION[\"Europe Equal Area 2001\",\n        METHOD[\"Lambert Azimuthal Equal Area\",\n            ID[\"EPSG\",9820]],\n        PARAMETER[\"Latitude of natural origin\",52,\n            ANGLEUNIT[\"degree\",0.0174532925199433],\n            ID[\"EPSG\",8801]],\n        PARAMETER[\"Longitude of natural origin\",10,\n            ANGLEUNIT[\"degree\",0.0174532925199433],\n            ID[\"EPSG\",8802]],\n        PARAMETER[\"False easting\",4321000,\n            LENGTHUNIT[\"metre\",1],\n            ID[\"EPSG\",8806]],\n        PARAMETER[\"False northing\",3210000,\n            LENGTHUNIT[\"metre\",1],\n            ID[\"EPSG\",8807]]],\n    CS[Cartesian,2],\n        AXIS[\"northing (Y)\",north,\n            ORDER[1],\n            LENGTHUNIT[\"metre\",1]],\n        AXIS[\"easting (X)\",east,\n            ORDER[2],\n            LENGTHUNIT[\"metre\",1]],\n    USAGE[\n        SCOPE[\"Statistical analysis.\"],\n        AREA[\"Europe - European Union (EU) countries and candidates. Europe - onshore and offshore: Albania; Andorra; Austria; Belgium; Bosnia and Herzegovina; Bulgaria; Croatia; Cyprus; Czechia; Denmark; Estonia; Faroe Islands; Finland; France; Germany; Gibraltar; Greece; Hungary; Iceland; Ireland; Italy; Kosovo; Latvia; Liechtenstein; Lithuania; Luxembourg; Malta; Monaco; Montenegro; Netherlands; North Macedonia; Norway including Svalbard and Jan Mayen; Poland; Portugal including Madeira and Azores; Romania; San Marino; Serbia; Slovakia; Slovenia; Spain including Canary Islands; Sweden; Switzerland; Türkiye (Turkey); United Kingdom (UK) including Channel Islands and Isle of Man; Vatican City State.\"],\n        BBOX[24.6,-35.58,84.73,44.83]],\n    ID[\"EPSG\",3035]]",
    "proj:epsg":3035,
    "proj:projjson":{
      "$schema":"https://proj.org/schemas/v0.7/projjson.schema.json",
      "type":"ProjectedCRS",
      "name":"ETRS89-extended / LAEA Europe",
      "base_crs":{
        "name":"ETRS89",
        "datum_ensemble":{
          "name":"European Terrestrial Reference System 1989 ensemble",
          "members":[
            {
              "name":"European Terrestrial Reference Frame 1989",
              "id":{
                "authority":"EPSG",
                "code":1178
              }
            },
            {
              "name":"European Terrestrial Reference Frame 1990",
              "id":{
                "authority":"EPSG",
                "code":1179
              }
            },
            {
              "name":"European Terrestrial Reference Frame 1991",
              "id":{
                "authority":"EPSG",
                "code":1180
              }
            },
            {
              "name":"European Terrestrial Reference Frame 1992",
              "id":{
                "authority":"EPSG",
                "code":1181
              }
            },
            {
              "name":"European Terrestrial Reference Frame 1993",
              "id":{
                "authority":"EPSG",
                "code":1182
              }
            },
            {
              "name":"European Terrestrial Reference Frame 1994",
              "id":{
                "authority":"EPSG",
                "code":1183
              }
            },
            {
              "name":"European Terrestrial Reference Frame 1996",
              "id":{
                "authority":"EPSG",
                "code":1184
              }
            },
            {
              "name":"European Terrestrial Reference Frame 1997",
              "id":{
                "authority":"EPSG",
                "code":1185
              }
            },
            {
              "name":"European Terrestrial Reference Frame 2000",
              "id":{
                "authority":"EPSG",
                "code":1186
              }
            },
            {
              "name":"European Terrestrial Reference Frame 2005",
              "id":{
                "authority":"EPSG",
                "code":1204
              }
            },
            {
              "name":"European Terrestrial Reference Frame 2014",
              "id":{
                "authority":"EPSG",
                "code":1206
              }
            },
            {
              "name":"European Terrestrial Reference Frame 2020",
              "id":{
                "authority":"EPSG",
                "code":1382
              }
            }
          ],
          "ellipsoid":{
            "name":"GRS 1980",
            "semi_major_axis":6378137,
            "inverse_flattening":298.257222101
          },
          "accuracy":"0.1",
          "id":{
            "authority":"EPSG",
            "code":6258
          }
        },
        "coordinate_system":{
          "subtype":"ellipsoidal",
          "axis":[
            {
              "name":"Geodetic latitude",
              "abbreviation":"Lat",
              "direction":"north",
              "unit":"degree"
            },
            {
              "name":"Geodetic longitude",
              "abbreviation":"Lon",
              "direction":"east",
              "unit":"degree"
            }
          ]
        },
        "id":{
          "authority":"EPSG",
          "code":4258
        }
      },
      "conversion":{
        "name":"Europe Equal Area 2001",
        "method":{
          "name":"Lambert Azimuthal Equal Area",
          "id":{
            "authority":"EPSG",
            "code":9820
          }
        },
        "parameters":[
          {
            "name":"Latitude of natural origin",
            "value":52,
            "unit":"degree",
            "id":{
              "authority":"EPSG",
              "code":8801
            }
          },
          {
            "name":"Longitude of natural origin",
            "value":10,
            "unit":"degree",
            "id":{
              "authority":"EPSG",
              "code":8802
            }
          },
          {
            "name":"False easting",
            "value":4321000,
            "unit":"metre",
            "id":{
              "authority":"EPSG",
              "code":8806
            }
          },
          {
            "name":"False northing",
            "value":3210000,
            "unit":"metre",
            "id":{
              "authority":"EPSG",
              "code":8807
            }
          }
        ]
      },
      "coordinate_system":{
        "subtype":"Cartesian",
        "axis":[
          {
            "name":"Northing",
            "abbreviation":"Y",
            "direction":"north",
            "unit":"metre"
          },
          {
            "name":"Easting",
            "abbreviation":"X",
            "direction":"east",
            "unit":"metre"
          }
        ]
      },
      "scope":"Statistical analysis.",
      "area":"Europe - European Union (EU) countries and candidates. Europe - onshore and offshore: Albania; Andorra; Austria; Belgium; Bosnia and Herzegovina; Bulgaria; Croatia; Cyprus; Czechia; Denmark; Estonia; Faroe Islands; Finland; France; Germany; Gibraltar; Greece; Hungary; Iceland; Ireland; Italy; Kosovo; Latvia; Liechtenstein; Lithuania; Luxembourg; Malta; Monaco; Montenegro; Netherlands; North Macedonia; Norway including Svalbard and Jan Mayen; Poland; Portugal including Madeira and Azores; Romania; San Marino; Serbia; Slovakia; Slovenia; Spain including Canary Islands; Sweden; Switzerland; Türkiye (Turkey); United Kingdom (UK) including Channel Islands and Isle of Man; Vatican City State.",
      "bbox":{
        "south_latitude":24.6,
        "west_longitude":-35.58,
        "north_latitude":84.73,
        "east_longitude":44.83
      },
      "id":{
        "authority":"EPSG",
        "code":3035
      }
    },
    "proj:transform":[
      2515000.0,
      100.0,
      0.0,
      5512800.0,
      0.0,
      -100.0
    ],
    "raster:bands":[
      {
        "data_type":"uint16",
        "nodata":null
      }
    ],
    "eo:bands":[
      {
        "name":"b1",
        "description":"Gray"
      }
    ]
  }
}

from datetime import datetime
import pystac

spatial_extent = pystac.SpatialExtent(bboxes=[[4,51,5,52]])
collection_interval = sorted([datetime(2020,1,1), datetime(2022,1,1)])
temporal_extent = pystac.TemporalExtent(intervals=[collection_interval])

description_markdown = """
SOC content in the 0-20 cm top soil expressed in g kg-1 for 100 m resolution pixels

Applications:

- First globally consistent and contiguous complete gridded soil property map of Europe
- Indicator of soil health, as per Mission Area Soil Health and Food
- By 2030, at least 75% of soils in each EU Member State should be in healthy condition or show a
significant improvement towards meeting accepted thresholds of indicators, to support ecosystem
services

*Reliability*: More than 83 % of the cross-validation points fall within the 70% prediction interval for the bare soil model.
For the vegetated area model 94 % of the points fall within the 90 % prediction interval.
"""

item_assets = {
    "SOC": {
       "type": "image/tiff; application=geotiff; profile=cloud-optimized",
        "title": "SOC",
        "description": "SOC content in the 0-20 cm top soil expressed in g kg-1",
        "data_type": "uint16",
        "nodata": 65535,
        "unit": "g kg-1",
        "roles": [
            "data"
        ]
    }
}

collection_extent = pystac.Extent(spatial=spatial_extent, temporal=temporal_extent)
collection = pystac.Collection(id='esa-worldsoils',
                               description=description_markdown,
                               extent=collection_extent,
                               extra_fields=dict(item_assets=item_assets),
                               license='CC-BY-SA-4.0')

collection.summaries.add("proj:code",["EPSG:3035"])
collection.summaries.add("gsd",[100])
collection.summaries.add("bands",[{
    "title": "SOC",
    "gsd": 100
}])


collection

• type "Collection"
• id "esa-worldsoils"
• stac_version "1.0.0"
• description " SOC content in the 0-20 cm top soil expressed in g kg-1 for 100 m resolution pixels Applications: - First globally consistent and contiguous complete gridded soil property map of Europe - Indicator of soil health, as per Mission Area Soil Health and Food - By 2030, at least 75% of soils in each EU Member State should be in healthy condition or show a significant improvement towards meeting accepted thresholds of indicators, to support ecosystem services *Reliability*: More than 83 % of the cross-validation points fall within the 70% prediction interval for the bare soil model. For the vegetated area model 94 % of the points fall within the 90 % prediction interval. "
• links[] 0 items
• item_assets

  • SOC

    • type "image/tiff; application=geotiff; profile=cloud-optimized"
    • title "SOC"
    • description "SOC content in the 0-20 cm top soil expressed in g kg-1"
    • data_type "uint16"
    • nodata 65535
    • unit "g kg-1"
    • roles[] 1 items

      • 0 "data"
• extent

  • spatial

    • bbox[] 1 items

      • 0[] 4 items

        • 0 4

        • 1 51

        • 2 5

        • 3 52
  • temporal

    • interval[] 1 items

      • 0[] 2 items

        • 0 "2020-01-01T00:00:00Z"

        • 1 "2022-01-01T00:00:00Z"
• license "CC-BY-SA-4.0"
• summaries

  • proj:code[] 1 items

    • 0 "EPSG:3035"
  • gsd[] 1 items

    • 0 100
  • bands[] 1 items

    • 0

      • title "SOC"
      • gsd 100

coll_dict = collection.to_dict()

default_auth = {
    "_auth": {
        "read": ["anonymous"],
        "write": ["stac-openeo-admin", "stac-openeo-editor"]
    }
}

coll_dict.update(default_auth)

response = requests.post("https://catalogue.project-a.apex.esa.int/collections", auth=auth,json=coll_dict)
response

<Response [201]>

import shapely


geometry = {
    "type":"Polygon",
    "coordinates":[
      [
        [
          -35.0421748,
          67.1549879
        ],
        [
          -9.3033749,
          33.067326
        ],
        [
          34.1876844,
          31.8603398
        ],
        [
          62.8622654,
          64.3475852
        ],
        [
          -35.0421748,
          67.1549879
        ]
      ]
    ]
  }
item_bbox = shapely.geometry.shape(geometry).bounds
collection_item = pystac.Item(id='europe_aggr-orgc_00-020_mean_100_202003-202210',
                              geometry=geometry,
                              bbox = item_bbox,
                              datetime=datetime(2022,3,1),
                              collection=collection.id,
                              properties={})

collection_item.common_metadata.gsd = 100

asset = pystac.Asset(href="https://eoresults.esa.int/d/APEX_TEST/2020/03/01/europe_aggr-orgc_00-020_mean_100_202003-202210/europe_aggr-orgc_00-020_mean_100_202003-202210.tif", 
                      media_type=pystac.MediaType.GEOTIFF, roles=["data"])
collection_item.add_asset("SOC", asset)
collection_item

• type "Feature"
• stac_version "1.0.0"
• id "europe_aggr-orgc_00-020_mean_100_202003-202210"
• properties

  • gsd 100
  • datetime "2022-03-01T00:00:00Z"
• geometry

  • type "Polygon"
  • coordinates[] 1 items

    • 0[] 5 items

      • 0[] 2 items

        • 0 -35.0421748

        • 1 67.1549879

      • 1[] 2 items

        • 0 -9.3033749

        • 1 33.067326

      • 2[] 2 items

        • 0 34.1876844

        • 1 31.8603398

      • 3[] 2 items

        • 0 62.8622654

        • 1 64.3475852

      • 4[] 2 items

        • 0 -35.0421748

        • 1 67.1549879
• links[] 0 items
• assets

  • SOC

    • href "https://eoresults.esa.int/d/APEX_TEST/2020/03/01/europe_aggr-orgc_00-020_mean_100_202003-202210/europe_aggr-orgc_00-020_mean_100_202003-202210.tif"
    • type "image/tiff; application=geotiff"
    • roles[] 1 items

      • 0 "data"
• bbox[] 4 items

  • 0 -35.0421748

  • 1 31.8603398

  • 2 62.8622654

  • 3 67.1549879
• stac_extensions[] 0 items
• collection "esa-worldsoils"

r.collection_item_create(collection.id, collection_item.to_dict())

True

Cleaning up

When the demo is done, we can simply clean up by deleting our (useless) test collection.

requests.delete("https://stac-openeo-dev.vgt.vito.be/collections/" + collection.id, auth=auth)

<Response [204]>