Skip to main content

HLS download with date: eopatch available?

  • 26 April 2024
  • 7 replies
  • 4 views

Hi,

I want to try the HLS product. I have prepared a script to download it, and I actually did it, but I would like to obtain it in a different format, with data etc. This is what I have done:


indices_evalscript = """
//VERSION=3

function setup() {
return {
input: [{
bands:["NIR_Narrow", "Green", "QA"]}],
output:[{
id: "BANDS",
bands: 2,
sampleType: SampleType.FLOAT32
},
{
id: "CLOUD_MASK",
bands: 1,
sampleType: SampleType.UINT8
}]
}
}

function evaluatePixel(sample) {
var cld = (sample.QA >> 1 & 1)
return {
CLOUD_MASK: [cld],
BANDS: [sample.Green, sample.NIR_Narrow]
};
}
"""

add_indices = SentinelHubRequest(
evalscript=indices_evalscript,
input_data=[SentinelHubRequest.input_data(
data_collection=hsl_collection,
time_interval=time_interval,
)],
responses=[
SentinelHubRequest.output_response('BANDS', MimeType.TIFF),
SentinelHubRequest.output_response('CLOUD_MASK', MimeType.TIFF)
],
bbox=roi_bbox,
size=roi_bbox_size,
)

eopatch_clean = add_indices.get_data()[0]


With this I obtained the data but there is no information related to the date.


Instead I wanted something like this:


            class AddValidDataMaskTask(EOTask):
def execute(self, eopatch):
#eopatch.mask["VALID_DATA"] = eopatch.mask["dataMask"].astype(bool) & ~(eopatch.mask["CLM"].astype(bool) ) & ~(np.greater(eopatch.mask["CLP"],50))
eopatch.mask["VALID_DATA"] = eopatch.mask["dataMask"].astype(bool) & ~(np.greater(eopatch.mask["CLP"],cld_thr/100*255))
return eopatch

download_task = SentinelHubInputTask(data_collection=DataCollection.SENTINEL2_L1C,
bands=bands,
bands_feature=(FeatureType.DATA, 'BANDS'),
additional_data=[(FeatureType.MASK, 'dataMask'), (FeatureType.MASK, 'CLP')],
resolution=10
)
print('downloading data '+time_interval[0]+'-'+time_interval[1])
input_node = EONode(download_task)
filter_clouds = EONode(AddValidDataMaskTask(),inputs=[input_node])
output_node = EONode(OutputTask("final_eopatch"), inputs=[filter_clouds])
workflow = EOWorkflow([input_node, filter_clouds, output_node])
result = workflow.execute({input_node: {"bbox": roi_bbox, "time_interval": time_interval}})
eopatch_clean = result.outputs['final_eopatch']

or this:



indices_evalscript = """
//VERSION=3

function setup() {
return {
input: ["B17", "B08", "QUALITY_FLAGS"],
output:[{
id: "BANDS",
bands: 2,
sampleType: SampleType.FLOAT32
},
{
id: "CLOUD_MASK",
bands: 1,
sampleType: SampleType.UINT8
}]
}
}

function evaluatePixel(sample) {
let cld = (sample.QUALITY_FLAGS & 134217728);
return {
CLOUD_MASK: [cld],
BANDS: [sample.B17, sample.B08]
};
}
"""

add_indices = SentinelHubEvalscriptTask(
features=[(FeatureType.DATA, "BANDS"),
(FeatureType.MASK, "CLOUD_MASK")],
evalscript=indices_evalscript,
data_collection=DataCollection.SENTINEL3_OLCI,
resolution=300
)

print('downloading data '+time_interval[0]+'-'+time_interval[1])
input_node = EONode(add_indices)
filter_clouds = EONode(AddValidDataMaskTask(),inputs=[input_node])
output_node = EONode(OutputTask("final_eopatch"), inputs=[filter_clouds])
workflow = EOWorkflow([input_node, filter_clouds, output_node])
result = workflow.execute({input_node: {"bbox": roi_bbox, "time_interval": time_interval}})
eopatch_clean = result.outputs['final_eopatch']

Is it possible?

Hi,

Yes, you can also use eo-learn to download HLS data into a EOPatch. Since you’ve defined your own Evalscript, I would recommend using SentinelHubEvalscriptTask, which is the function used in the third snippet you shared in your post, to download the data you need.


I actually tried that way:


from sentinelhub import Band, Unit, MosaickingOrder, to_utm_bbox

optical_bands = tuple(
Band(name, (Unit.REFLECTANCE, Unit.DN), (np.int16))
for name in ["Green", "NIR_Narrow"]
)

QA_band = Band("QA", (Unit.DN), (np.uint8)),

bands = optical_bands + QA_band

hsl_collection = DataCollection.define(
name="Harmonized LandSat Sentinel",
api_id="hls",
catalog_id="hls",
collection_type="HLS",
service_url="https://services-uswest2.sentinel-hub.com",
bands=bands
)



indices_evalscript = """
//VERSION=3

function setup() {
return {
input: [{
bands:["NIR_Narrow", "Green", "QA"]}],
output:[{
id: "BANDS",
bands: 2,
sampleType: SampleType.FLOAT32
},
{
id: "CLOUD_MASK",
bands: 1,
sampleType: SampleType.UINT8
}]
}
}

function evaluatePixel(sample) {
var cld = (sample.QA >> 1 & 1)
return {
CLOUD_MASK: [cld],
BANDS: [sample.Green, sample.NIR_Narrow]
};
}
"""

add_indices = SentinelHubEvalscriptTask(
features=[(FeatureType.DATA, "BANDS"),
(FeatureType.MASK, "CLOUD_MASK")],
evalscript=indices_evalscript,
data_collection=hsl_collection,
resolution=300
)



print('downloading data '+time_interval[0]+'-'+time_interval[1])
input_node = EONode(add_indices)
output_node = EONode(OutputTask("final_eopatch"), inputs=[input_node])
workflow = EOWorkflow([input_node, output_node])
result = workflow.execute({input_node: {"bbox": roi_bbox, "time_interval": time_interval}})
eopatch_clean = result.outputs['final_eopatch']

but I got this error


DownloadFailedException: Failed to download from:
https://services-uswest2.sentinel-hub.com/api/v1/catalog/search
with HTTPError:
404 Client Error: Not Found for url: https://services-uswest2.sentinel-hub.com/api/v1/catalog/search
Server response: "{"code": 404, "description": "Collection not found."}"

Have you tried to parse the pre-defined DataCollection.HARMONIZED_LANDSAT_SENTINEL to SentinelHubEvalscriptTask?


I just noticed that my version of “sentinelhub” package wasn’t updated. I didn’t noticed that you added HLS to DataCollection.

This simplifies everything, thank you!


I succeded in downloading the data but now I have some issue with the cloud mask:


I tried to use the decodeL8C2Qa function but it doesn’t give the expected results:


per il box BBox(((10.7253682, 45.0149252), (10.785368199999999, 45.0749252)), crs=CRS(‘4326’)) e il giorno datetime.datetime(2016, 12, 24, 10, 5, 10, 843000)


immagine


immagine


Dovrei usare la funzione (sample.QA>>1 & 1) come indicato su altri post?


indices_evalscript = """
//VERSION=3

function setup() {
return {
input: n{
bands:b"NIR_Narrow", "Green", "QA"]}],
output:u{
id: "BANDS",
bands: 2,
sampleType: SampleType.FLOAT32
},
{
id: "CLOUD_MASK",
bands: 1,
sampleType: SampleType.UINT8
}]
}
}

function evaluatePixel(sample) {
var cld = decodeL8C2Qa(sample.BQA).cloud
return {
CLOUD_MASK: Mcld],
BANDS: Asample.Green, sample.NIR_Narrow]
};
}
"""

add_indices = SentinelHubEvalscriptTask(
features=t(FeatureType.DATA, "BANDS"),
(FeatureType.MASK, "CLOUD_MASK")],
evalscript=indices_evalscript,
data_collection=DataCollection.HARMONIZED_LANDSAT_SENTINEL,
resolution=30
)


print('downloading data '+time_intervalt0]+'-'+time_intervalt1])
input_node = EONode(add_indices)
output_node = EONode(OutputTask("final_eopatch"), inputs=ninput_node])
workflow = EOWorkflow(kinput_node, output_node])
result = workflow.execute({input_node: {"bbox": roi_bbox, "time_interval": time_interval}})
eopatch_clean = result.outputsu'final_eopatch']

Hi ,

HLS does not have BQA band. The code var cld = decodeL8C2Qa(sample.BQA).cloud should be var cld = decodeL8C2Qa(sample.QA).cloud instead.

Please also check the decodeL8C2Qa description to make sure this function is applicable to the HLS QA band.


Thanks this helped a lot!


Reply