Skip to main content

BYOC with CREODIAS Bucket

  • 26 April 2024
  • 7 replies
  • 25 views

I am following the instructions on https://docs.sentinel-hub.com/api/latest/api/byoc/#byoc-deployment


From here How to generate ec2 credentials? - FAQ Networking - CREODIAS


  • I had to enable openstack_waw3_1 location

  • downloaded the openrc.sh file with the configurations

  • I got my ec2 credentials

    In WAW-3:

  • I created a bucket (named OTvhr)

  • uploaded my cog files: tile_1/XXX.tiff and tile_2/YYY.tiff

    From here https://docs.sentinel-hub.com/api/latest/af3077d767c7d1b7cfc39fd52e838270/creodias_set_bucket_access_policy.py:

  • I downloaded the script and completed access_key, secret_key and bucket_name.

  • I had to set host=‘https://s3.waw3-1.cloudferro.com/’ as the original value gave me “botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the PutBucketPolicy operation: Unknown” as an error.

    I configured the Sentinel Hub collection:

  • using my project id from openrc.sh file and the bucket name

  • collection location is correctly identified as CREODIAS

  • tiles ingestion fails with: Failed to access the storage.

The only thing I can think of that I did that was not on the documentation is changing the using s3.waw3-1.cloudferro.com instead of s3.waw2-1.cloudferro.com. Is this the issue? How can I correctly configure the buckets?


Thanks


Regards

Hi,

Sentinel Hub doesn’t yet support s3.waw3-1.cloudferro.com endpoint. Any chance you could use a bucket in s3.waw2-1.cloudferro.com cloud meanwhile. No ETA yet when waw3-1 will be supported.

apologies for the inconveniences


Thanks, I managed to create a bucket on waw2, but now I get:


“Found bands with invalid names that cannot be used in evalscripts”, should I open another thread?


That means the band name contains invalid characters. The documentation indicates that:



The band name should be a valid JavaScript identifier so it can be safely used in evalscripts; valid identifiers are case-sensitive, can contain Unicode letters, $, _, and digits (0-9), but may not start with a digit, and should not be one of the reserved JavaScript keywords.



There are several options to solve this


(1) You could rename you files to valid names based on the above constraints or


(2) You could manually configure your bands directly in the collection before ingesting the tiles. There is an option to configure bands. The manual band configuration can also be done using the API


If you still don’t solve it, please paste here an example of your tile path to help easily identify the error


Thanks, I got it working with your suggestions.


I am now faced with a render issue.
image


Images on the bucket are OK.


I can open the images and get decent renderization from QGIS without tweaking enhancements.


gdalinfo of one of the tiles:


gdalinfo byoc_tiles/PleiadesNeo_Balcarce_pansharp_1_1/B.tiff 
Driver: GTiff/GeoTIFF
Files: byoc_tiles/PleiadesNeo_Balcarce_pansharp_1_1/B.tiff
Size is 13832, 15360
Coordinate System is:
GEOGCRS["WGS 84",
DATUM["World Geodetic System 1984",
ELLIPSOID["WGS 84",6378137,298.257223563,
LENGTHUNIT["metre",1]]],
PRIMEM["Greenwich",0,
ANGLEUNIT["degree",0.0174532925199433]],
CS[ellipsoidal,2],
AXIS["geodetic latitude (Lat)",north,
ORDER[1],
ANGLEUNIT["degree",0.0174532925199433]],
AXIS["geodetic longitude (Lon)",east,
ORDER[2],
ANGLEUNIT["degree",0.0174532925199433]],
ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Origin = (-58.289053000000003,-37.815680000000000)
Pixel Size = (0.000002699902400,-0.000002699912435)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
COMPRESSION=DEFLATE
INTERLEAVE=PIXEL
LAYOUT=COG
PREDICTOR=2
Corner Coordinates:
Upper Left ( -58.2890530, -37.8156800) ( 58d17'20.59"W, 37d48'56.45"S)
Lower Left ( -58.2890530, -37.8571507) ( 58d17'20.59"W, 37d51'25.74"S)
Upper Right ( -58.2517079, -37.8156800) ( 58d15' 6.15"W, 37d48'56.45"S)
Lower Right ( -58.2517079, -37.8571507) ( 58d15' 6.15"W, 37d51'25.74"S)
Center ( -58.2703805, -37.8364153) ( 58d16'13.37"W, 37d50'11.10"S)
Band 1 Block=1024x1024 Type=UInt16, ColorInterp=Gray
NoData Value=0
Overviews: 6916x7680, 3458x3840, 1729x1920, 864x960
Band 2 Block=1024x1024 Type=UInt16, ColorInterp=Undefined
NoData Value=0
Overviews: 6916x7680, 3458x3840, 1729x1920, 864x960
Band 3 Block=1024x1024 Type=UInt16, ColorInterp=Undefined
NoData Value=0
Overviews: 6916x7680, 3458x3840, 1729x1920, 864x960
Band 4 Block=1024x1024 Type=UInt16, ColorInterp=Undefined
NoData Value=0
Overviews: 6916x7680, 3458x3840, 1729x1920, 864x960
Band 5 Block=1024x1024 Type=UInt16, ColorInterp=Undefined
NoData Value=0
Overviews: 6916x7680, 3458x3840, 1729x1920, 864x960
Band 6 Block=1024x1024 Type=UInt16, ColorInterp=Undefined
NoData Value=0
Overviews: 6916x7680, 3458x3840, 1729x1920, 864x960

And the evalscript used:


//VERSION=3

let minVal = 0.1;
let maxVal = 0.11;

let viz = new HighlightCompressVisualizer(minVal, maxVal);

function setup() {
return {
input: [{
bands: [
"B1",
"B2",
"B3",
"dataMask"
]
}],
output: {
bands: 4,
sampleType: "AUTO"
}
}
}

function evaluatePixel(sample) {
let val = [sample.B3/65535, sample.B2/65535, sample.B1/65535];
val = viz.processList(val);
val.push(sample.dataMask);
return val;
}

I really tried a lot of variants, but never got a decent render.


Thanks in advance!

Regards.


Hi,

Would you mind letting us know what bands you’re using, i.e., what are the B1 to B7 in your collection? Also, could you please share a screenshot of the expected visualisation?

Thank you!


Hi,

The expected visualization is True Color using Bands 1,2,3 as R,G,B.

This screenshot is generated using QGIS and the pre-uploaded files.

Bands 4 and + are not used.

I just caught that the script invert the bands order, but using them in the right order gives the same kind of unwanted result.

Regards.


I recreated the collection, but I did not rename the bands (the automatic nomenclature was B_1, B_2, etc. and I was renaming them to B1, B2, etc.), this time it worked.


Does anyone know why was this happening?


Regards


Reply