Solved

Daily product in production

  • 1 November 2022
  • 7 replies
  • 209 views
J
Userlevel 1
Badge +3
  • jacob
  • ‎‎ 🌱
  • 2 posts replied to

We are building an automated daily product based on the analytic_8b_udm2 bundle and having trouble coming up with a process for quick search + ordering assets. We recognize there is a processing time and that time varies. Is there anything in the API to know if an AOI doesn’t have that asset available for that day at all?  Or is currently processing?

icon

Best answer by cholmes 4 November 2022, 16:26

View original

7 replies

Joel Wolfson
Rank
Badge +4

@cholmes

cholmes
Rank
Userlevel 4
Badge +4

Hey @jacob - thanks for your question. Unfortunately right now there’s nothing in the API to know if an asset is available or is currently processing. Usually the actual ‘processing’ step is relatively short (certainly under an hour), most of the time in the delay is getting the imagery downloaded from the satellite. The one exception is the processing time of surface reflectance data, which depends on atmospheric information from the MODIS satellites, so that can be up to 12 hours delay I believe. But you’d see the analytic asset, the surface reflectance would show up when available. 

I hope that helps. I do think there’s probably more we could do to communicate that an asset likely won’t be in the area of interest for the day, but it’s a pretty tricky problem to get right.

best regards,

Chris

cholmes
Rank
Userlevel 4
Badge +4

Also, @jacob - have you considered using the subscriptions API? It’ll auto-order any asset matching your filters, so you’ll be sure to have the latest images as soon as they’re available.

J
Userlevel 1
Badge +3

Also, @jacob - have you considered using the subscriptions API? It’ll auto-order any asset matching your filters, so you’ll be sure to have the latest images as soon as they’re available.

 

@cholmes 

Interesting, we were under the impression that the subscription API delivered assets at a much slower pace (deliveries made once a week is what we thought) and that searching for assets and ordering them through the orders API would get the assets in our hands faster.  I’ve been wondering how this was going to scale for us with thousands of AOIs given the request rate limiting.

Are you saying that it would in fact shorten the asset delivery time for us to use the subscription API rather than polling the quick search API and then ordering the assets with the orders API?

cholmes
Rank
Userlevel 4
Badge +4

Interesting, we were under the impression that the subscription API delivered assets at a much slower pace (deliveries made once a week is what we thought)

Hrm, no - deliveries are made as soon as the asset is available - subscriptions were designed to obviate the need for polling. We actually don’t even have the ability to make deliveries once a week through the subscriptions API (though have talked about it)

Are you saying that it would in fact shorten the asset delivery time for us to use the subscription API rather than polling the quick search API and then ordering the assets with the orders API?

 

Yes, exactly. Most users with thousands of AOI’s who want updates use subscriptions. It’ll easily handle tens of thousands of AOI’s, delivering the assets as soon as we have them available.

Note I would recommend using ortho_analytic_8b_sr for most analytic use cases, see the Surface Reflectance white paper for more information. It won’t show up quite as fast, but will usually be ready within a day of capture. It’s also good to use the harmonization tool, with the Sentinel 2 target, for the most consistent stacks of data. 

 

J
Userlevel 1
Badge +3

Subscription sounds perfect. We’ll try it! 

Note I would recommend using ortho_analytic_8b_sr for most analytic use cases, see the Surface Reflectance white paper for more information. It won’t show up quite as fast, but will usually be ready within a day of capture. It’s also good to use the harmonization tool, with the Sentinel 2 target, for the most consistent stacks of data. 

Actually I pasted the wrong bundle!  We are using that SR product with the analytic_8b_sr_udm2 bundle. Last time we looked at delays, acquired to published date, we saw times around 7 to 30 hour delays with some less frequent case over 100 hours.

cholmes
Rank
Userlevel 4
Badge +4

Sounds great on using subscriptions. 

Ah, that’s great you’re using SR already. And yes, that delay from acquired to published sounds about right. I think there may have been some recent improvements that get the vast majority published within 24 hours. And yes, there are some edge cases that can lead to longer delays. 

 

Reply


Terms & Conditions