Page 4 of 4

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Sat Jan 23, 2021 6:33 am
by sekellyd7
Hi

I have used the subset_dataset.py script (here: https://github.com/nasa/podaac_tools_an ... dataset.py) to subset from MUR-JPL-L4-GLOB-v4.1 recently (~October 2020) and was able to download the subsetted data without issue

However I have tried today using the same script and same python version and receive the following error:

Traceback (most recent call last):
File "./subset_dataset.py", line 542, in <module>
standalone_main()
File "./subset_dataset.py", line 235, in standalone_main
response = urllib.request.urlopen(wsurl)
File "//anaconda3/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "//anaconda3/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "//anaconda3/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "//anaconda3/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "//anaconda3/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "//anaconda3/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Unknown Reason

I am unsure as to how to go about fixing this error or is it something to do with the URL? Any advice would be much appreciated and thank you for making these subset tools readily available.

Many thanks
Sean

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Mon Jan 25, 2021 3:58 pm
by yiboj
Hi Sean,
Thanks for your inquiry.
I just run the test on my system using anaconda3 python 3.7 and it works, not sure exactly what wrong with your runs. Could you please run other dataset to see if it works or not ? Please let me know.
Regards,
-PODAAC DE

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Tue Jan 26, 2021 2:14 am
by sekellyd7
Hi

Thanks for you reply. I have run it again exactly as before and now its working perfectly (for MUR-JPL-L4-GLOB-v4.1)!!

Thanks
Sean

ARIF VIANSAH

PostPosted: Wed Mar 03, 2021 11:44 am
by arifviansah0
:)
yiboj wrote:Hi Sean,
Thanks for your inquiry.
I just run the test on my system using anaconda3 python 3.7 and it works, not sure exactly what wrong with your runs. Could you please run other dataset to see if it works or not ? Please let me know.
Regards,
-PODAAC DE
sekellyd7 wrote:Hi

I have used the subset_dataset.py script (here: https://github.com/nasa/podaac_tools_an ... dataset.py) to subset from MUR-JPL-L4-GLOB-v4.1 recently (~October 2020) and was able to download the subsetted data without issue

However I have tried today using the same script and same python version and receive the following error:

Traceback (most recent call last):
File "./subset_dataset.py", line 542, in <module>
standalone_main()
File "./subset_dataset.py", line 235, in standalone_main
response = urllib.request.urlopen(wsurl)
File "//anaconda3/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "//anaconda3/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "//anaconda3/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "//anaconda3/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "//anaconda3/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "//anaconda3/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Unknown Reason

I am unsure as to how to go about fixing this error or is it something to do with the URL? Any advice would be much appreciated and thank you for making these subset tools readily available.

Many thanks
Sean

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Wed Jun 23, 2021 6:51 pm
by davidusta
Hi !. Thank you so much for the code. My question is if we can use this piece of code on a Jupyter Notebook file. I am starting with the process of automatically download data but I dont know if this is possible.

Thanks in advance!.

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Fri Jun 25, 2021 3:42 am
by davidusta
I actually try to download the AQUARIUS_L3_SSS_RAINCORRECTED_CAP_7DAY_V5 file

%run subset_data.py -s 20120801 -f 20120101 -b -130 -80 -50 -20 -x AQUARIUS_L3_SSS_RAINCORRECTED_CAP_7DAY_V5

And an error was showed:

SystemExit: No granules found for dataset: AQUARIUS_L3_SSS_RAINCORRECTED_CAP_7DAY_V5
Program will exit now !