STAC collection or catalog for multiple year data

[ivorbosloper]

Quite some data sets provide multiple years of data. See for instance https://source.coop/fiboa/be-vlg

In the current publication process, the convert command has an option --collection which generates a STAC collection.json file for a single data set. We publish the data (parquet+pmtiles) and the stac/collection.json file to set to source coop, see this example's be-vlg collection.json.

What should we do with multiple years? We could keep a stac/collection.json file pointing to the most recent parquet/pmtiles file. But can we add a stac/catalog.json file pointing to all files, or should that be in the same collection-file?

@m-mohr you probably have an idea about this...

[m-mohr]

We'd probably add multiple temporal extents to the collection and also add multiple assets to the collection, one per year. We could also create Items per year, depends a bit on our usecase, but for the map we'd currently need everything in one file.

I assume we'd need to read from an existing collection.json in case we want to add additional years.

[m-mohr]

@ivorbosloper wrote in chat:

When I opened this issue #150 , I had this case in mind: https://source.coop/fiboa/at-crop
So there’s a source.coop directory with multiple parquet files and multiple stac files. Is there a way to ‘merge’ the stac files into one file? Should that be a catalog?
I’m probably asking a very basic STAC question, just trying to understand the spec…

I responded as follows:

---

I can imagine it's not so easy to navigate the spec, especially as we are currently doing it a bit different compared to the STAC best practices.
We pretty much create only Collections that point to Parquet files while usually that should be Items.
Comes a bit from the fact that we didn't really create a full STAC catalog in the first place, but just combine various separate things with a single collection usually.
Normally we'd do a Collection fiboa that points to items for each dataset, but we now do a Catalog fiboa that points to collections instead.
It depends a bit how we want it to behave now. I think the map currently only works on the collections, so that may need a bit of an update.
I think for the inidivual years we should start to use items. So the structure could look like this:

• fiboa (Catalog)
  • at_crop (Collection, multiple years)
    • 2021 (Item)
    • 2022 (Item)
    • ...
  • de_bb (Collection, only one year)
  • ...

So create items once we have multiple years, keep it in the collection for one year. In the long term we should probably also create items for single-year datasets, but that's future work, I guess.
The CLI will change with regards to STAC Collection creation anyway, I think I'd make it a separate command that's not part of the convert process and then we can also improve the whole structure that we create.