Use FIS to only get dates (without stats)?

Hi,

We have a dedicated service that responds perfectly to your needs! 😎

To request only dates of available products for a BBOX and with a MAXCC parameter, you can use the Catalog API. This will avoid you from fetching unnecessary data (in your case) that comes with the FIS requests.

There are a number of examples in the documentation, but if you get stuck don’t hesitate to post in the forum again.

Thanks!

Looks like the feature I needed is the Catalog API with the “distinct” option, which returns a JSON file that include a nice list of dates of the available products in a YYYY-MM-DD format.

I have two requests:

• I was surprised to notice that adding “distinct” : “date” returns the dates in a chronological order (older products first) while without it we get products in reverse order. Is there an option to turn this around? (that distinct returns the date list newest first).
• the “limit” parameter has a maximum value of 100, could you enable to raise it to 1000 or more when using the “distinct” option? I am aware of the “next” option, but it is just a list of dates, it feels that the system will not collapse by requesting a list of 1000 elements in YYYY-MM-DD format.

A third request: Is there a way of using “distinct”:“date” along with a “maxcc” parameter? Otherwise I am just getting an exhaustive list of products date but I cannot easily filter out the cloudy ones.

To filter the clouds, you can use the query parameter in addition to distinct. For example with Sentinel-2 L2A images (accessible parameters with Catalog API), you can specify:

"query": {"eo:cloud_cover": {"lt": 10}}

for images less than 10% cloudy.

I was surprised to notice that adding “distinct” : “date” returns the dates in a chronological order (older products first) while without it we get products in reverse order. Is there an option to turn this around? (that distinct returns the date list newest first).

This inconsistent behaviour should not happen and the development team will look into this. However, I don’t think this will be addressed shortly. I would recommend you sort the output in the mean time.

the “limit” parameter has a maximum value of 100, could you enable to raise it to 1000 or more when using the “distinct” option?

This option is not supported, and we therefore suggest to use pagination.

Hi

thanks for your answers! I could get working a javascript piece of code that retrieved the dates I wanted, but harcdoging a temporary access token.

Now the problem is that this token already expired, as opposed to the instance ID used in OGC API which is permanent.

I saw the docs about authentication and using OAuth to generate a new access token from javascript, but I am wondering why it is more cumbersome to get a list of dates than to do a FIS request? Is it possible to access the Catalog API with an Instance ID?

Hey,

if you are using npm or yarn for managing the packages for your project, I suggest using sentinelhub-js which simplifies the usage of Sentinel Hub services.

The updating of the auth token still needs to be done by hand, but it’s easier as it is just a call of a function with client id and secret to request a new token. Checking if token has expired is not part of the library yet.

const authToken = await requestAuthToken(clientId, clientSecret);
setAuthToken(authToken);

It’s also easier to get all the dates, because it enables getting them with or without the auth token. Also, the pagination is used if dates are requested on the Catalog API, so you get all the results with 1 function call.

The library also simplifies getting the images either via WMS or via Processing API.

The most basic setup is presented in the example below (example contains random parameters).
This example doesn’t use Catalog API to get the dates, so Catalog API features are not available here.

import { S2L2ALayer, BBox, MimeTypes, ApiType } from '@sentinel-hub/sentinelhub-js';

const instanceId = ...;
const layerId = ...;
const layer = new S2L2ALayer({
 instanceId,
 layerId,
 maxCloudCoverPercent: 60,
});
const bbox4326 = new BBox(CRS_EPSG4326, 11.9, 42.05, 12.95, 43.09);
const fromTime = new Date(Date.UTC(2020, 1 - 1, 1, 0, 0, 0));
const toTime = new Date(Date.UTC(2020, 1 - 1, 15, 23, 59, 59));

// getting dates
const dates = await layer.findDatesUTC(bbox4326, fromTime, toTime);

// getting images
 const getMapParams = {
   bbox: bbox4326,
   fromTime: fromTime,
   toTime: toTime,
   width: 512,
   height: 512,
   format: MimeTypes.JPEG,
   }
 };
const imageBlob = await layer.getMap(getMapParams, ApiType.WMS);

• To use features of Catalog API and Processing API, the token should be set, but the function calls don’t change, only setAuthToken(authToken); should be added before any function, that needs authentication, is called. There is also a function to check if the token was already set, to not call the setAuthToken unnecessarily.

The library is not yet ready to be used via CDN, so in case you are not using npm or yarn, here are some tips.

• Requesting a new token is pretty easy to write in javascript and can be packed into a utility function (example in the docs). The solution in sentinelhub-js is a bit different from the example.

  async function requestAuthToken(clientId, clientSecret) {
    const response = await axios({
      method: 'post',
      url: 'https://services.sentinel-hub.com/oauth/token',
      headers: { 'content-type': 'application/x-www-form-urlencoded' },
      data: `grant_type=client_credentials&client_id=${encodeURIComponent(
        clientId,
      )}&client_secret=${encodeURIComponent(clientSecret)}`,
    });
    return response.data.access_token;
  }
  

• To go through the paginated dates, wrap the request in a while loop and check if the response has a field context.next (break the loop if the field is not present).

Is it possible to access the Catalog API with an Instance ID?

Unfortunately not when using Catalog API directly, but it can be when using sentinelhub-js (auth token should be still requested and set, but from then on, the layer object can be created only with instance id and layer id).

I am wondering why it is more cumbersome to get a list of dates than to do a FIS request?

The decision to protect some features with authentication (and some also with pagination) was made to prevent abuse and consequent unwanted downtime or slow response since some requests are processed in real time and require more processing power. Getting dates is an example of such request.

Hope this answers as many questions as possible.

Hi,

thanks a lot for your answer, now everything works!

Indeed I am not using NPM/YARN, only CDN, so your piece of javascript code really saved me 🙂

I would suggest that this piece of code replace the javascript example on the Authentication page of the Sentinel-Hub doc, since it does not require qs/querystring, which is an unnecessary dependency in this context (in a CDN context it seems almost impossible to make it work).

Another question related to the Catalog API: when accessing the sentinel-3-olci catalog, there is no way to indicate a maximum cloud cover in the query. Is there some workaround for this?

I believe (but not 100% sure) that Sentinel-3 does not have cloud cover meta-data.