Page 3 of 4

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Thu Apr 04, 2019 2:40 am
by gssanjeev_126
I am unable to locate subset_dataset.py file.
Can you please provide the python script.

My interest is SMAP L2 dataset which is in .h5 file type

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Mon Apr 08, 2019 7:49 am
by yiboj
Thank you for your inquiry.

Here is the link to the ftp site, please check.

ftp://podaac.jpl.nasa.gov/allData/commo ... dataset.py

Thank you for your patience.

PODAAC Team

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Mon Apr 08, 2019 8:13 pm
by gssanjeev_126
Thank you for your response.

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Fri May 17, 2019 12:24 pm
by sal13
Hello. Thank you very much for the code to download a subset from large gridded datasets. I am new to Python and I am trying to download from the OSTM_L2_OST_OGDR_GPS. I tried the following command:
python subset_dataset_l2.py -s 20090601 -f 20190501 -b -90 -60 10 30 -x OSTM_L2_OST_OGDR_GPS

Please wait while program searching for the granules ...


Longitude range: -90.000000 to -60.000000
Latitude range: 10.000000 to 30.000000

OK to download? [yes or no]: yes
Traceback (most recent call last):
File "subset_dataset_l2.py", line 266, in <module>
standalone_main()
File "subset_dataset_l2.py", line 233, in standalone_main
selindex = np.where( (lon > box[0]) & (lon < box[1]) & (lat > box[2]) & (lat < box[3]) )
TypeError: '>' not supported between instances of 'GridType' and 'float'

Can you guide me in how to fix this error? Thanks.

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Mon Jun 03, 2019 12:27 pm
by yiboj
Hi,

The current python code is using pydap package and it will not work with dataset OSTM_L2_OST_OGDR_GPS, please use the HiTIDE tool https://podaac-tools.jpl.nasa.gov/hitide/ to subset level 2 dataset for now.
Thanks,

PODAAC DE

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Wed Jun 12, 2019 4:21 am
by mcarr03
Hi I have an similar issue to the quoted below.

I have run the script successfully on a linux machine but wish run the code on a windows machine. Each time I run the script I get the error message below although I have wget installed. Has this query been answered?

Thanks

'The script will need curl or wget on the system, please install them first before running the script !
Program will exit now ![/code]'



rsdpc wrote:Hi, I using this script to subset data from AVHRR_OI-NCEI-L4-GLOB-v2.0. I enter this:

Code: Select all
subset_dataset.py -s 19810901 -f 20171014 -b -6 -5.5 36 36.2 -x


and receive the following result:

Code: Select all
Please wait while program searching for the granules ...
Longitude range: -6.000000 to -5.500000
Latitude range: 36.000000 to 36.200000
  every 1 pixel(s) is obtained

grid dimensions will be ( 4 x 2 )
OK to download?  [yes or no]: yes

The script will need curl or wget on the system, please install them first before running the script !
Program will exit now !


I have installed both wget and curl on the system (including them in PATH folder and as environment variables), and both of them work well. However the script doesn't seem to recognize they are installed somehow. Is there any solution for this? :? :?

Thanks in advance

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Mon Jun 17, 2019 8:53 pm
by yiboj
Hi,

Thanks you for your inquiry.
There are various fixes for the script to run in microsoft window OS, and here is the quick fix for it. Please check and let us know if it works for you.

Replace line 526-531:
Code: Select all
      if status_curl == 0:
        cmd='curl -g "'+cmd+'" -o '+ ncout
      elif status_wget == 0:
        cmd='wget "'+cmd+'" -O '+ ncout
      else:
        sys.exit('\nThe script will need curl or wget on the system, please install them first before running the script !\nProgram will exit now !\n')


with:
Code: Select all
      cmd='wget "'+cmd+'" -O '+ ncout


Regards,

PODAAC DE

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Tue Jun 18, 2019 12:20 pm
by sal13
Thank you for the suggestion to use the HiTIDE4 tool.. Worked perfectly!

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Sun Jul 21, 2019 11:15 am
by justcom
Hi
I couldn't find subset_dataset.py at the link you posted above
( ftp://podaac.jpl.nasa.gov/allData/commo ... dataset.py ) but dug around and found a version at https://podaac-tools.jpl.nasa.gov/drive ... subsetting
The comments in the file give the most recent edit at 2017.09.26. Is this the latest version?
Thanks!
Justin

Re: Using Python to subset large gridded datasets (v2)

PostPosted: Thu Jul 25, 2019 11:17 am
by yiboj
Hi Justin,

I have updated the link from ftp to PODAAC Drive in the post since the ftp is obsolete in PODAAC site, so you downloaded the correct file.
Regards,

PODAAC