Using Python to subset large gridded datasets (v2)

[sekellyd7]

Hi

I have used the subset_dataset.py script (here: https://github.com/nasa/podaac_tools_an ... dataset.py) to subset from MUR-JPL-L4-GLOB-v4.1 recently (~October 2020) and was able to download the subsetted data without issue

However I have tried today using the same script and same python version and receive the following error:

Traceback (most recent call last):
File "./subset_dataset.py", line 542, in <module>
standalone_main()
File "./subset_dataset.py", line 235, in standalone_main
response = urllib.request.urlopen(wsurl)
File "//anaconda3/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "//anaconda3/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "//anaconda3/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "//anaconda3/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "//anaconda3/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "//anaconda3/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Unknown Reason

I am unsure as to how to go about fixing this error or is it something to do with the URL? Any advice would be much appreciated and thank you for making these subset tools readily available.

Many thanks
Sean

[yiboj]

Hi Sean,
Thanks for your inquiry.
I just run the test on my system using anaconda3 python 3.7 and it works, not sure exactly what wrong with your runs. Could you please run other dataset to see if it works or not ? Please let me know.
Regards,
-PODAAC DE

[sekellyd7]

Hi

Thanks for you reply. I have run it again exactly as before and now its working perfectly (for MUR-JPL-L4-GLOB-v4.1)!!

Thanks
Sean

[arifviansah0]

Hi Sean,
Thanks for your inquiry.
I just run the test on my system using anaconda3 python 3.7 and it works, not sure exactly what wrong with your runs. Could you please run other dataset to see if it works or not ? Please let me know.
Regards,
-PODAAC DE

Hi

I have used the subset_dataset.py script (here: https://github.com/nasa/podaac_tools_an ... dataset.py) to subset from MUR-JPL-L4-GLOB-v4.1 recently (~October 2020) and was able to download the subsetted data without issue

However I have tried today using the same script and same python version and receive the following error:

Traceback (most recent call last):
File "./subset_dataset.py", line 542, in <module>
standalone_main()
File "./subset_dataset.py", line 235, in standalone_main
response = urllib.request.urlopen(wsurl)
File "//anaconda3/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "//anaconda3/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "//anaconda3/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "//anaconda3/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "//anaconda3/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "//anaconda3/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Unknown Reason

I am unsure as to how to go about fixing this error or is it something to do with the URL? Any advice would be much appreciated and thank you for making these subset tools readily available.

Many thanks
Sean

[davidusta]

Hi !. Thank you so much for the code. My question is if we can use this piece of code on a Jupyter Notebook file. I am starting with the process of automatically download data but I dont know if this is possible.

Thanks in advance!.

[davidusta]

I actually try to download the AQUARIUS_L3_SSS_RAINCORRECTED_CAP_7DAY_V5 file

%run subset_data.py -s 20120801 -f 20120101 -b -130 -80 -50 -20 -x AQUARIUS_L3_SSS_RAINCORRECTED_CAP_7DAY_V5

And an error was showed:

SystemExit: No granules found for dataset: AQUARIUS_L3_SSS_RAINCORRECTED_CAP_7DAY_V5
Program will exit now !

[komal_preet]

Hi, I am using the python code provided at PODAAC website to download data, however, I am getting this error even after making so many changes.

Shortname is required !
Program will exit now !

Usage: ipykernel_launcher.py [options]
-t CMD, -- cmd=CMD curl or wget
-x SHORTNAME, --JPL-L4UHfnd-GLOB-MUR=SHORTNAME
product short name
-s DATE0, --20100101=DATE0
start date: Format yyyymmdd (eg. -s 20140502 for May
2, 2014) [default: yesterday 20220216]
-f DATE1, --20110101=DATE1
finish date: Format yyyymmdd (eg. -f 20140502 for May
2, 2014) [default: today 20220217]
An exception has occurred, use %tb to see the full traceback.

SystemExit:
End date should be in format yyyymmdd !
Program will exit now !

[podaac]

Thank you for your inquiry.

Please try the following command first to see if it works and let us know if you have any further issues.

subset_dataset.py -s 20100101 -f 20100201 -b
-140 -110 20 30 -x MUR-JPL-L4-GLOB-v4.1