Problems with clouds / missing values when using WCS for larger areas (getting the right WCS parameters))

I am trying to download cloud-free Sentinel-2 bands “B03”,“B04”,“B05”,“B06”,“B07”,“B08”,“B11”,“B12” using WCS for the whole province of Alberta (Canada) at 20 m resolution; out of it I would like to make a cloud-free mosaic. I have made the following function (R) for this:

get_shub = function(i, tileS.tbl, COVERAGE="B03", TIME='2019-03-01/2019-06-01', RES=20, priority='leastCC', wcs.id="bec83f1b-4183-492d-b60a-d19408cc925d", out.dir="/data/Pachaterrae/data/shub"){

  out.png <- paste0(out.dir, "/T", tileS.tbl i,"ID"], "/T", tileS.tblli,"ID"], "_", COVERAGE, "_", plotKML::normalizeFilename(TIME), "_", RES,"m.png")

  bbox = paste(round(tileS.tbl>i,1:4],1), collapse = ",")

  if(!file.exists(out.png)){

require(curl)

Sys.sleep(.2)

x = download.file(paste0('https://services.sentinel-hub.com/ogc/wcs/', wcs.id, '?SERVICE=WCS&REQUEST=GetCoverage&COVERAGE=', COVERAGE, '&SHOWLOGO=false&MAXCC=5&priority=', priority, '&CRS=EPSG:3857&BBOX=', bbox, '&RESX=', RES, '&RESY=', RES,'&FORMAT=image/png&TIME=',TIME), out.png)

system(paste0('gdal_translate -of Gtiff -a_ullr ', paste0(round(tileS.tblai,c(1,4,3,2)],1), collapse = " "), ' -co \"COMPRESS=DEFLATE\" -a_srs \"EPSG:3857\" ', out.png, ' ', gsub(".png", ".tif", out.png)))

  }

}

I have run this function using tiling system (40 x 40 km) to avoid the WCS limit of 5000 x 5000 pixels and for various dates (TIME=‘2019-04-01/2019-06-01’, TIME=‘2018-05-01/2018-07-01’, TIME=‘2018-07-01/2018-09-01’). The resulting mosaic looks like this:

There are two problems with this mosaic:

A. For over 15% of the area I get missing pixels. Basically mosaic needs to be made again.

B. For about 10-20% of the area I still get clouds.

Note that for the WCS parameters I have used ‘MAXCC=5&priority=leastCC’ hoping that this would solve the problems of clouds.

My three questions are:

Have you maybe made some seasonal cloud-free products with Sentinel-2 bands that I could use? I am interested in 1-2 month periods in 2017-2019.

Note that I download the images as PNGs then I have to convert them to TIFs - is this the only way to get GeoTIFFs from the SH WCS? Which WCS parameters would you recommend to download cloud-free / missing-pixels free Sentinel 2 bands for a larger area?

The download speed using WCS is excellent and I managed to run the function above in parallel (in <8hrs it downloaded 400GB of data). This is fantastic - your WCS is amazing!! But my account limits me to 100,000 requests per month (is this correct?). That means that by running downloads for <24hrs would result in me using all my credits.

Maybe I should look at downloading all cloud-free scenes for a more narrow time period and then producing cloud-free / artifacts-free mosaics?

thanks in advance,

Tom Hengl

OpenGeoHub Foundation: Connect | Create | Share | Repeat – 17 Aug 20

About - OpenGeoHub Foundation: Connect | Create | Share | Repeat

We harness the power of Machine Learning and Geo-computations for Global Good, enabling and accelerating organizations in producing analysis-ready &

Page 1 / 1

tom.hengl:

1 Have you maybe made some seasonal cloud-free products with Sentinel-2 bands that I could use? I am interested in 1-2 month periods in 2017-2019.

2 Note that I download the images as PNGs then I have to convert them to TIFs - is this the only way to get GeoTIFFs from the SH WCS? Which WCS parameters would you recommend to download cloud-free / missing-pixels free Sentinel 2 bands for a larger area?

3 The download speed using WCS is excellent and I managed to run the function above in parallel (in <8hrs it downloaded 400GB of data). This is fantastic - your WCS is amazing!! But my account limits me to 100,000 requests per month (is this correct?). That means that by running downloads for <24hrs would result in me using all my credits.

No, we do unfortunately not build seasonal products ourselves as each user have their own requirements and it is best to leave it to the users. What I would recommend you do is to use AOI-based cloud detection first (use 100m resolution for cloud detection as it will be much faster and results will be asgood) and then use the date, which has least clouds.

Sure, you can get GeoTiffs out of the service, see
https://www.sentinel-hub.com/develop/documentation/api/output-formats

Note that you are using “individual account”, which is not really meant for such mass analysis (this is why there is not “machine 2 machine” check near this account type in pricing page). You should also note that it is 100.000 “processing units”, not 100.000 “individual requests”. In your case specifically you have executed 312.000 processing units in the last month (see this in your Dashboard). We do not yet enforce these limits too strictly, unless we see some user going over it too often. You should consider some of the larger plans. As you can imagine, it costs us something to run this WCS and the more users use it, more it costs us (processing and data transfer), which is why our pricing plans are configured based on the volume of requests.

I am most happy to hear you like our service!

Best,

Grega