subset_dataset.py download with curl????

subset_dataset.py download with curl????

Postby jashadehm » Wed Jun 02, 2021 5:39 pm

Hi,
I am stuck in a bit of a pickle. I hope you will be able to help me figure this out.

I am trying to download the sst .nc files for all days of the year, for the years 2003 to present from the MUR-JPL-L4-GLOB-v4.1 dataset

I have been attempting to download this using the subset_dataset.py code, as described in this forum post--> Using Python to subset large gridded datasets (v2) available here https://podaac.jpl.nasa.gov/forum/viewtopic.php?f=85&t=219

i run this file using the command
Code: Select all
python subset_dataset.py -s 20030101 -f 20210501 -b -140 -110 20 30 -x MUR-JPL-L4-GLOB-v4.1


it runs, it searches for granules, defines the grid dimension and then it asks me
OK to download? [yes or no]:
I respond with yes

And this is where it stops working. it says.
The script will need curl or wget on the system, please install them first before running the script !
Program will exit now !


The thing is I have curl (and I've tried with wget) on my python environment.

AND I know that curl is working. I know this because the drive_download.py described in this forum Multiple File Download from PO.DAAC Drive Using Python works. To run this script I use the command
Code: Select all
python drive_download.py -u USERNAME:PASSWORD -t curl -s 20030101 -f 20210501 -x MUR25-JPL-L4-GLOB-v04.2
..... unfortunately, using this is not an option for me as it downloads the entire data set for all-day for all years, which my internet will not download in a reasonable time frame.

What am I missing? What do I need to run the subset_dataset.py.. where do I insert my earth data login details, do I somehow merge both codes? OR can I define lat and lon in the drive_download.py

Alternatively, is there some other way I can download all these files in batch?

A super big thank you for helping me solve this....

Kind regards.
jashadehm
 
Posts: 1
Joined: Wed Jun 02, 2021 5:04 pm

Re: subset_dataset.py download with curl????

Postby yiboj » Sun Jun 06, 2021 6:19 pm

Hi,
Thanks for your inquiry and support.
In order to make your code to work, please insert the following line after line #545:

cmd='curl -g "'+cmd+'" -o '+ ncout


Hope this works for you and please let me know if you have any further questions.
Regards,

PODAAC DE
yiboj
 
Posts: 130
Joined: Mon Mar 30, 2015 11:22 am


Return to General Questions