Using Python to subset large gridded datasets (v2)

Re: Using Python to subset large gridded datasets (v2)

Postby sekellyd7 » Sat Jan 23, 2021 6:33 am

Hi

I have used the subset_dataset.py script (here: https://github.com/nasa/podaac_tools_an ... dataset.py) to subset from MUR-JPL-L4-GLOB-v4.1 recently (~October 2020) and was able to download the subsetted data without issue

However I have tried today using the same script and same python version and receive the following error:

Traceback (most recent call last):
File "./subset_dataset.py", line 542, in <module>
standalone_main()
File "./subset_dataset.py", line 235, in standalone_main
response = urllib.request.urlopen(wsurl)
File "//anaconda3/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "//anaconda3/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "//anaconda3/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "//anaconda3/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "//anaconda3/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "//anaconda3/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Unknown Reason

I am unsure as to how to go about fixing this error or is it something to do with the URL? Any advice would be much appreciated and thank you for making these subset tools readily available.

Many thanks
Sean
sekellyd7
 
Posts: 2
Joined: Sat Jan 23, 2021 6:19 am

Re: Using Python to subset large gridded datasets (v2)

Postby yiboj » Mon Jan 25, 2021 3:58 pm

Hi Sean,
Thanks for your inquiry.
I just run the test on my system using anaconda3 python 3.7 and it works, not sure exactly what wrong with your runs. Could you please run other dataset to see if it works or not ? Please let me know.
Regards,
-PODAAC DE
yiboj
 
Posts: 130
Joined: Mon Mar 30, 2015 11:22 am

Re: Using Python to subset large gridded datasets (v2)

Postby sekellyd7 » Tue Jan 26, 2021 2:14 am

Hi

Thanks for you reply. I have run it again exactly as before and now its working perfectly (for MUR-JPL-L4-GLOB-v4.1)!!

Thanks
Sean
sekellyd7
 
Posts: 2
Joined: Sat Jan 23, 2021 6:19 am

ARIF VIANSAH

Postby arifviansah0 » Wed Mar 03, 2021 11:44 am

:)
yiboj wrote:Hi Sean,
Thanks for your inquiry.
I just run the test on my system using anaconda3 python 3.7 and it works, not sure exactly what wrong with your runs. Could you please run other dataset to see if it works or not ? Please let me know.
Regards,
-PODAAC DE
sekellyd7 wrote:Hi

I have used the subset_dataset.py script (here: https://github.com/nasa/podaac_tools_an ... dataset.py) to subset from MUR-JPL-L4-GLOB-v4.1 recently (~October 2020) and was able to download the subsetted data without issue

However I have tried today using the same script and same python version and receive the following error:

Traceback (most recent call last):
File "./subset_dataset.py", line 542, in <module>
standalone_main()
File "./subset_dataset.py", line 235, in standalone_main
response = urllib.request.urlopen(wsurl)
File "//anaconda3/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "//anaconda3/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "//anaconda3/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "//anaconda3/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "//anaconda3/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "//anaconda3/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Unknown Reason

I am unsure as to how to go about fixing this error or is it something to do with the URL? Any advice would be much appreciated and thank you for making these subset tools readily available.

Many thanks
Sean
arifviansah0
 
Posts: 1
Joined: Wed Mar 03, 2021 11:04 am

Re: Using Python to subset large gridded datasets (v2)

Postby davidusta » Wed Jun 23, 2021 6:51 pm

Hi !. Thank you so much for the code. My question is if we can use this piece of code on a Jupyter Notebook file. I am starting with the process of automatically download data but I dont know if this is possible.

Thanks in advance!.
davidusta
 
Posts: 2
Joined: Wed Jun 23, 2021 6:49 pm

Re: Using Python to subset large gridded datasets (v2)

Postby davidusta » Fri Jun 25, 2021 3:42 am

I actually try to download the AQUARIUS_L3_SSS_RAINCORRECTED_CAP_7DAY_V5 file

%run subset_data.py -s 20120801 -f 20120101 -b -130 -80 -50 -20 -x AQUARIUS_L3_SSS_RAINCORRECTED_CAP_7DAY_V5

And an error was showed:

SystemExit: No granules found for dataset: AQUARIUS_L3_SSS_RAINCORRECTED_CAP_7DAY_V5
Program will exit now !
davidusta
 
Posts: 2
Joined: Wed Jun 23, 2021 6:49 pm

Previous

Return to Data Access and Services