Training/Inference deep learning models fully in the cloud

Hello

suppose I have developed locally a python workflow for training and inference of a deep learning model using planetscope data(using sample data).

Now I want to move everything to the cloud considering NoR protocols.

When I read the docs i couldn’t find out how i should do this process fully in the cloud?

should I rewrite the code in evalscript?

(i though i might be able to package the code in a container and upload it to the sentinel-hub server, but i could n’t find anything related to containers)

Thanks in advance for help

Regards

Page 1 / 1

Hi @tom.holland1392,

For the time being (we are working on something, but will take time), Sentinel Hub doesn’t take care of the cloud infrastructure for you, meaning you have two options:

you re-write your processing (ML, DL, …) in evalscript (which is pixel-based!)

you use the Sentinel Hub services to retrieve the data and do the rest on the cloud by managing your own infrastructure

The latter can be a bit simplified by using eo-learn and eo-grow. Particularly with eo-grow, the infrastructure management is a bit easier; you can have a look at GEM example, and the instructions on how to use eo-grow with Ray for large scale processing.

Regarding the former, please have a look at ML as custom script example from eo-learn to see if it is feasible in your case.

Best,

Matej

Thanks @batic for your helpful answer.

i have some comments regarding two, which would be grateful if you confirm or correct :

** you re-write your processing (ML, DL, …) in evalscript (which is pixel-based!)*

1 - we can only use pre-trained simple machine learning models like DT (which can be stringified easily). we can not train even those simple models.

2 - for more complicated models like DL witch usually get implemented using Tensorflow or pytorch libraries, we can not even use (inference) the pre-trained models(because it is too much hard and error prune to convert these models to evalscript if be possible). if it is not correct is there any example of using DL as an evalscript?

** you use the Sentinel Hub services to retrieve the data and do the rest on the cloud by managing your own infrastructure*

in this case although all processing are done in the cloud, but it is outside the sentinel-hub. I mean it is similar to do the task in my own infrastructure in local system. in the local system we also download the data and do the rest processing steps. in this option i also use my own infrastructure but in the cloud (AWS/Google cloud/…).

Thanks for your help.

Hi @batic

As you said that you are working on something but it would last time, I am eager to know if there is any update regarding this?

Regards,

Tom