I originally raised this question on the Sentinel-Hub forum under the Sentinel@AWS category, but got no replies. Perhaps I’ll get more luck here?
It seems that several S2 L2A products with baseline 05.00 (collection 1) are missing from the AWS bucket sentinel-s2-l2a.
Let’s take as an example product S2B_MSIL2A_20220108T174719_N0500_R098_T13SER_20230205T190405:
It is available on CDSE.
But it is not on AWS, instead only the older version of the product, with baseline 03.01, is there (s3://sentinel-s2-l2a/products/2022/1/8/S2B_MSIL2A_20220108T174719_N0301_R098_T13SER_20220108T214244/)
This is the case for many, if not all, collection 1 products from 2022.
Are there any plans to add these products on AWS?
Many thanks Carlo
Best answer by williamray
Hi, I can only comment on what has been stated on the Github page link that you attached in your previous message: the missing products are a known issue and that reprocessing of the archive is an ongoing task:
However, as of April 2024, the ESA reprocessing of the entire archive to baseline 5.0 is incomplete, so the time periods of Nov 2016 to Nov 2019 and 2022 are missing. It is unknown when this process will be completed.
For more information on the status of Sentinel-2 reprocessing, you can visit this page.
We do not routinely update the archive with reprocessed data, as we do not directly support accessing the data directly from the S3 Bucket. This means we cannot guarantee that reprocessed data will be added to the archive in Cloud Optimized GeoTIFF format. For comprehensive access to the imagery we recommend you use the CDSE platform.
Unfortunately, we are unable to support to replicated Sentinel-2 data on the AWS bucket. This means in rare examples like yours the reprocessed data would not be available to access.
I would recommend using CDSE as the primary source of your Sentinel-2 L2A products if it is important to you to have the latest version of the product. There is also a separate deployment of Sentinel Hub on the CDSE platform that you can use instead.
Unfortunately my example is not rare. Hundreds of thousands of Sentinel-2 collection-1 products are missing from the AWS buckets, for both L1C and L2A collections.
Hi, I can only comment on what has been stated on the Github page link that you attached in your previous message: the missing products are a known issue and that reprocessing of the archive is an ongoing task:
However, as of April 2024, the ESA reprocessing of the entire archive to baseline 5.0 is incomplete, so the time periods of Nov 2016 to Nov 2019 and 2022 are missing. It is unknown when this process will be completed.
For more information on the status of Sentinel-2 reprocessing, you can visit this page.
We do not routinely update the archive with reprocessed data, as we do not directly support accessing the data directly from the S3 Bucket. This means we cannot guarantee that reprocessed data will be added to the archive in Cloud Optimized GeoTIFF format. For comprehensive access to the imagery we recommend you use the CDSE platform.
The information quoted from Element84’s Github dates back to April 2024. Since then, ESA completed the reprocessing of the entire archive to baseline 5.0, as indicated in the CDSE documentation:
Updated availability by sensing time period
Sentinel-2A
Sentinel-2B
Published (Processing baseline 05.00)
From 4 July 2015 to 31 December 2021 included
From 17 March 2017 to 31 December 2021 included
However, most of these products have not been ingested on AWS. Here are a few examples of products found on CDSE that are not available on AWS: S2A_MSIL2A_20160108T201602_N0500_R142_T01CCV_20231014T062746 S2A_MSIL2A_20160103T210522_N0500_R071_T01CCV_20231010T092948 S2A_MSIL2A_20160107T204522_N0500_R128_T01CCV_20231008T064310 S2A_MSIL2A_20160110T205522_N0500_R028_T01CCV_20231008T111031 S2A_MSIL2A_20160104T203522_N0500_R085_T01CCV_20231009T185852
Hence the question is: does Planet/Sinergise still manage the Sentinel-2 AWS buckets? If yes, which rules do determine which Sentinel-2 products are replicated to AWS?
the Collection-1 fully replaces the products processed with the previous baselines. We are now proceeding with the removal of the old baseline Sentinel-2 data with sensing date up to 31 December 2021 from the Copernicus Data Space Ecosystem.
The deletion process will start on 24 October and is estimated to be completed by 15 November.
AWS Sentinel-2 buckets will then be completely out of sync with CDSE for 2015 - 2023.
