Using Python to subset large gridded datasets (v2)

Re: Using Python to subset large gridded datasets (v2)

Postby gssanjeev_126 » Thu Apr 04, 2019 2:40 am

I am unable to locate subset_dataset.py file.
Can you please provide the python script.

My interest is SMAP L2 dataset which is in .h5 file type
gssanjeev_126
 
Posts: 2
Joined: Thu Apr 04, 2019 2:35 am

Re: Using Python to subset large gridded datasets (v2)

Postby yiboj » Mon Apr 08, 2019 7:49 am

Thank you for your inquiry.

Here is the link to the ftp site, please check.

ftp://podaac.jpl.nasa.gov/allData/commo ... dataset.py

Thank you for your patience.

PODAAC Team
yiboj
 
Posts: 88
Joined: Mon Mar 30, 2015 11:22 am

Re: Using Python to subset large gridded datasets (v2)

Postby gssanjeev_126 » Mon Apr 08, 2019 8:13 pm

Thank you for your response.
gssanjeev_126
 
Posts: 2
Joined: Thu Apr 04, 2019 2:35 am

Re: Using Python to subset large gridded datasets (v2)

Postby sal13 » Fri May 17, 2019 12:24 pm

Hello. Thank you very much for the code to download a subset from large gridded datasets. I am new to Python and I am trying to download from the OSTM_L2_OST_OGDR_GPS. I tried the following command:
python subset_dataset_l2.py -s 20090601 -f 20190501 -b -90 -60 10 30 -x OSTM_L2_OST_OGDR_GPS

Please wait while program searching for the granules ...


Longitude range: -90.000000 to -60.000000
Latitude range: 10.000000 to 30.000000

OK to download? [yes or no]: yes
Traceback (most recent call last):
File "subset_dataset_l2.py", line 266, in <module>
standalone_main()
File "subset_dataset_l2.py", line 233, in standalone_main
selindex = np.where( (lon > box[0]) & (lon < box[1]) & (lat > box[2]) & (lat < box[3]) )
TypeError: '>' not supported between instances of 'GridType' and 'float'

Can you guide me in how to fix this error? Thanks.
sal13
 
Posts: 2
Joined: Fri May 17, 2019 12:18 pm

Re: Using Python to subset large gridded datasets (v2)

Postby yiboj » Mon Jun 03, 2019 12:27 pm

Hi,

The current python code is using pydap package and it will not work with dataset OSTM_L2_OST_OGDR_GPS, please use the HiTIDE tool https://podaac-tools.jpl.nasa.gov/hitide/ to subset level 2 dataset for now.
Thanks,

PODAAC DE
yiboj
 
Posts: 88
Joined: Mon Mar 30, 2015 11:22 am

Re: Using Python to subset large gridded datasets (v2)

Postby mcarr03 » Wed Jun 12, 2019 4:21 am

Hi I have an similar issue to the quoted below.

I have run the script successfully on a linux machine but wish run the code on a windows machine. Each time I run the script I get the error message below although I have wget installed. Has this query been answered?

Thanks

'The script will need curl or wget on the system, please install them first before running the script !
Program will exit now ![/code]'



rsdpc wrote:Hi, I using this script to subset data from AVHRR_OI-NCEI-L4-GLOB-v2.0. I enter this:

Code: Select all
subset_dataset.py -s 19810901 -f 20171014 -b -6 -5.5 36 36.2 -x


and receive the following result:

Code: Select all
Please wait while program searching for the granules ...
Longitude range: -6.000000 to -5.500000
Latitude range: 36.000000 to 36.200000
  every 1 pixel(s) is obtained

grid dimensions will be ( 4 x 2 )
OK to download?  [yes or no]: yes

The script will need curl or wget on the system, please install them first before running the script !
Program will exit now !


I have installed both wget and curl on the system (including them in PATH folder and as environment variables), and both of them work well. However the script doesn't seem to recognize they are installed somehow. Is there any solution for this? :? :?

Thanks in advance
mcarr03
 
Posts: 1
Joined: Wed Jun 12, 2019 4:08 am

Re: Using Python to subset large gridded datasets (v2)

Postby yiboj » Mon Jun 17, 2019 8:53 pm

Hi,

Thanks you for your inquiry.
There are various fixes for the script to run in microsoft window OS, and here is the quick fix for it. Please check and let us know if it works for you.

Replace line 526-531:
Code: Select all
      if status_curl == 0:
        cmd='curl -g "'+cmd+'" -o '+ ncout
      elif status_wget == 0:
        cmd='wget "'+cmd+'" -O '+ ncout
      else:
        sys.exit('\nThe script will need curl or wget on the system, please install them first before running the script !\nProgram will exit now !\n')


with:
Code: Select all
      cmd='wget "'+cmd+'" -O '+ ncout


Regards,

PODAAC DE
yiboj
 
Posts: 88
Joined: Mon Mar 30, 2015 11:22 am

Re: Using Python to subset large gridded datasets (v2)

Postby sal13 » Tue Jun 18, 2019 12:20 pm

Thank you for the suggestion to use the HiTIDE4 tool.. Worked perfectly!
sal13
 
Posts: 2
Joined: Fri May 17, 2019 12:18 pm

Re: Using Python to subset large gridded datasets (v2)

Postby justcom » Sun Jul 21, 2019 11:15 am

Hi
I couldn't find subset_dataset.py at the link you posted above
( ftp://podaac.jpl.nasa.gov/allData/commo ... dataset.py ) but dug around and found a version at https://podaac-tools.jpl.nasa.gov/drive ... subsetting
The comments in the file give the most recent edit at 2017.09.26. Is this the latest version?
Thanks!
Justin
justcom
 
Posts: 1
Joined: Sun Jul 21, 2019 9:47 am

Re: Using Python to subset large gridded datasets (v2)

Postby yiboj » Thu Jul 25, 2019 11:17 am

Hi Justin,

I have updated the link from ftp to PODAAC Drive in the post since the ftp is obsolete in PODAAC site, so you downloaded the correct file.
Regards,

PODAAC
yiboj
 
Posts: 88
Joined: Mon Mar 30, 2015 11:22 am

Previous

Return to Data Access and Services

cron