Issue in downloading S2 L1C images

I am downloading S2 L1c data in several area worldwide. In most of them the data are obtained without issue, but in one area I found some errors.

Here is the code:

coordstat=[127.39,  70.68]



area1=[coordstat[0]-0.02,coordstat[1]-0.02,coordstat[0]+0.02,coordstat[1]+0.02]



roi_bbox = BBox(bbox=area1, crs=CRS.WGS84)



time_interval = (str(2021)+'-10-01T00:00:00.000000000', str(years[-1]+2)+'-09-30T23:59:59.000000000')

class AddValidDataMaskTask(EOTask):

    def execute(self, eopatch):

        #eopatch.mask["VALID_DATA"] = eopatch.mask["dataMask"].astype(bool) & ~(eopatch.mask["CLM"].astype(bool) ) & ~(np.greater(eopatch.mask["CLP"],50))

        eopatch.mask["VALID_DATA"] = eopatch.mask["dataMask"].astype(bool) & ~(np.greater(eopatch.mask["CLP"],cld_thr/100*255))

        return eopatch



download_task = SentinelHubInputTask(data_collection=DataCollection.SENTINEL2_L1C,

                                     bands=bands,

                                     bands_feature=(FeatureType.DATA, 'BANDS'),

                                     additional_data=[(FeatureType.MASK, 'dataMask'), (FeatureType.MASK, 'CLP')],

                                     resolution=10

                                    )

print('downloading data '+time_interval[0]+'-'+time_interval[1])

input_node = EONode(download_task)

filter_clouds = EONode(AddValidDataMaskTask(),inputs=[input_node])

output_node = EONode(OutputTask("final_eopatch"), inputs=[filter_clouds])

workflow = EOWorkflow([input_node, filter_clouds, output_node])

result = workflow.execute({input_node: {"bbox": roi_bbox, "time_interval": time_interval}})

eopatch_clean = result.outputs['final_eopatch']

This is the error that I obtained:

DownloadFailedException: Failed to download from:

https://services.sentinel-hub.com/api/v1/process

with HTTPError:

500 Server Error: Internal Server Error for url: https://services.sentinel-hub.com/api/v1/process

Server response: "{"status": 500, "reason": "Internal Server Error", "message": "Illegal request to s3://sentinel-s2-l1c-index/tiles/51/W/XU/2023/3/10/0/B08.index. HTTP Status: '404'", "code": "RENDERER_EXCEPTION"}"

DownloadFailedException: Failed to download from:

https://services.sentinel-hub.com/api/v1/process

with HTTPError:

500 Server Error: Internal Server Error for url: https://services.sentinel-hub.com/api/v1/process

Server response: "{"status": 500, "reason": "Internal Server Error", "message": "Illegal request to s3://sentinel-s2-l1c-index/tiles/51/W/XU/2023/3/10/0/B08.index. HTTP Status: '404'", "code": "RENDERER_EXCEPTION"}"

Page 1 / 1

I obtained something similar in another station:

coordstat=[-152.12, 69.36]



area1=[coordstat[0]-0.02,coordstat[1]-0.02,coordstat[0]+0.02,coordstat[1]+0.02]



roi_bbox = BBox(bbox=area1, crs=CRS.WGS84)



time_interval = (str(2016)+'-10-01T00:00:00.000000000', str(2023)+'-09-30T23:59:59.000000000')

class AddValidDataMaskTask(EOTask):

    def execute(self, eopatch):

        #eopatch.mask["VALID_DATA"] = eopatch.mask["dataMask"].astype(bool) & ~(eopatch.mask["CLM"].astype(bool) ) & ~(np.greater(eopatch.mask["CLP"],50))

        eopatch.mask["VALID_DATA"] = eopatch.mask["dataMask"].astype(bool) & ~(np.greater(eopatch.mask["CLP"],cld_thr/100*255))

        return eopatch



download_task = SentinelHubInputTask(data_collection=DataCollection.SENTINEL2_L1C,

                                     bands=bands,

                                     bands_feature=(FeatureType.DATA, 'BANDS'),

                                     additional_data=[(FeatureType.MASK, 'dataMask'), (FeatureType.MASK, 'CLP')],

                                     resolution=10

                                    )

print('downloading data '+time_interval[0]+'-'+time_interval[1])

input_node = EONode(download_task)

filter_clouds = EONode(AddValidDataMaskTask(),inputs=[input_node])

output_node = EONode(OutputTask("final_eopatch"), inputs=[filter_clouds])

workflow = EOWorkflow([input_node, filter_clouds, output_node])

result = workflow.execute({input_node: {"bbox": roi_bbox, "time_interval": time_interval}})

eopatch_clean = result.outputs['final_eopatch']

And I got a similar error

DownloadFailedException: Failed to download from:

https://services.sentinel-hub.com/api/v1/process

with HTTPError:

500 Server Error: Internal Server Error for url: https://services.sentinel-hub.com/api/v1/process

Server response: "{"status": 500, "reason": "Internal Server Error", "message": "Illegal request to s3://sentinel-s2-l1c-index/tiles/5/W/NS/2023/4/7/0/B08.index. HTTP Status: '404'", "code": "RENDERER_EXCEPTION"}"

Hi,

Thank you for reporting the issue. I’ve forwarded the issue to our developers and will let you know when the issue is fixed.

Hi,

The issue should be fixed. Please try again and let us know if the issue persists.

Hi,

It has been solved, but I found another one with S3:

DownloadFailedException: Failed to download from:

https://creodias.sentinel-hub.com/api/v1/process

with HTTPError:

500 Server Error: Internal Server Error for url: https://creodias.sentinel-hub.com/api/v1/process

Server response: "{"status": 500, "reason": "Internal Server Error", "message": "Illegal request to creo://EODATA/Sentinel-3/OLCI/OL_1_EFR/2017/12/01/S3A_OL_1_EFR____20171201T040107_20171201T040407_20171202T082843_0179_025_104_2520_LN1_O_NT_002.SEN3/geo_coordinates.nc. HTTP Status: '404' On CreoDIAS, you get 'Illegal request 403 error' also when a file is missing. So first make sure that the file is present.", "code": "RENDERER_EXCEPTION"}"

Hi,

It seems this file EODATA/Sentinel-3/OLCI/OL_1_EFR/2017/12/01/S3A_OL_1_EFR____20171201T040107_20171201T040407_20171202T082843_0179_025_104_2520_LN1_O_NT_002.SEN3/geo_coordinates.nc is missing in the bucket.

I would recommend reporting the issue to CreoDIAS support.