Recursive/successive calls to to Solr via WS

Recursive/successive calls to to Solr via WS

Postby lmcgibbn » Tue Apr 25, 2017 9:40 pm

Hi Folks,
I previously discovered that I am not able to submit a single query containing an array of DatasetId's (this was covered in https://podaac.jpl.nasa.gov/forum/viewtopic.php?f=53&t=558). As a workaround, I am therefore submitting recursive/successive calls which enable me to retrieve individual GCMD records. Whenever the PO.DAAC dataset suite is updated, I will be running a request against every dataset and retrieving the GCMD manifestation. I'll therefore be making some ~560 requests and I would like to do this with as little latency between calls as possible.
From my experience administering Web crawlers, I am aware of politeness measures which can be taken to limit the load on servers. Can someone please advise a reasonable politeness time period I can stick to which will keep everyone happy? I do not want to annoy anyone.
Thank you very much in advance.
Lewis
lmcgibbn
 
Posts: 56
Joined: Wed Feb 17, 2016 9:36 pm

Return to PO.DAAC Web Services

cron