Hi,

Unfortunately, we are unable to support to replicated Sentinel-2 data on the AWS bucket. This means in rare examples like yours the reprocessed data would not be available to access.

I would recommend using CDSE as the primary source of your Sentinel-2 L2A products if it is important to you to have the latest version of the product. There is also a separate deployment of Sentinel Hub on the CDSE platform that you can use instead.

Hi William,

Thank you for replying.

Unfortunately my example is not rare. Hundreds of thousands of Sentinel-2 collection-1 products are missing from the AWS buckets, for both L1C and L2A collections.

As a consequence, these missing products are not transformed to Cloud-optimized GeoTIFFs (COGs) by Element 84, which is a pity since JP2 does not allow efficient access to small windows from rasters through http range requests.

Here is a list of 8000 L2A products that are available on CDSE but not on AWS:

https://pastebin.com/TA2LfLrF

It stops at 8000 items to stay below the 512 KB allowed on pastebin, but the full list is several hundreds of thousands.

Hi,
I can only comment on what has been stated on the Github page link that you attached in your previous message: the missing products are a known issue and that reprocessing of the archive is an ongoing task:

However, as of April 2024, the ESA reprocessing of the entire archive to baseline 5.0 is incomplete, so the time periods of Nov 2016 to Nov 2019 and 2022 are missing. It is unknown when this process will be completed.

For more information on the status of Sentinel-2 reprocessing, you can visit this page.

We do not routinely update the archive with reprocessed data, as we do not directly support accessing the data directly from the S3 Bucket. This means we cannot guarantee that reprocessed data will be added to the archive in Cloud Optimized GeoTIFF format. For comprehensive access to the imagery we recommend you use the CDSE platform.

Hi William,

The information quoted from Element84’s Github dates back to April 2024. Since then, ESA completed the reprocessing of the entire archive to baseline 5.0, as indicated in the CDSE documentation:

Updated availability by sensing time period	Sentinel-2A	Sentinel-2B
Published (Processing baseline 05.00)	From 4 July 2015 to 31 December 2021 included	From 17 March 2017 to 31 December 2021 included

However, most of these products have not been ingested on AWS. Here are a few examples of products found on CDSE that are not available on AWS:
S2A_MSIL2A_20160108T201602_N0500_R142_T01CCV_20231014T062746 S2A_MSIL2A_20160103T210522_N0500_R071_T01CCV_20231010T092948 S2A_MSIL2A_20160107T204522_N0500_R128_T01CCV_20231008T064310 S2A_MSIL2A_20160110T205522_N0500_R028_T01CCV_20231008T111031 S2A_MSIL2A_20160104T203522_N0500_R085_T01CCV_20231009T185852

Hence the question is: does Planet/Sinergise still manage the Sentinel-2 AWS buckets? If yes, which rules do determine which Sentinel-2 products are replicated to AWS?
 

CDSE announced yesterday that reprocessing of the archive to baseline 05.00 is completed and old baseline data will be removed: https://dataspace.copernicus.eu/news/2024-10-10-sentinel-2-old-baselines-products-deletion
 

the Collection-1 fully replaces the products processed with the previous baselines. We are now proceeding with the removal of the old baseline Sentinel-2 data with sensing date up to 31 December 2021 from the Copernicus Data Space Ecosystem.

The deletion process will start on 24 October and is estimated to be completed by 15 November.

AWS Sentinel-2 buckets will then be completely out of sync with CDSE for 2015 - 2023.