Page 1 of 1

Recursive/successive calls to to Solr via WS

PostPosted: Tue Apr 25, 2017 9:40 pm
by lmcgibbn
Hi Folks,
I previously discovered that I am not able to submit a single query containing an array of DatasetId's (this was covered in As a workaround, I am therefore submitting recursive/successive calls which enable me to retrieve individual GCMD records. Whenever the PO.DAAC dataset suite is updated, I will be running a request against every dataset and retrieving the GCMD manifestation. I'll therefore be making some ~560 requests and I would like to do this with as little latency between calls as possible.
From my experience administering Web crawlers, I am aware of politeness measures which can be taken to limit the load on servers. Can someone please advise a reasonable politeness time period I can stick to which will keep everyone happy? I do not want to annoy anyone.
Thank you very much in advance.