Page 1 of 1

Using Python to Subset and Download Level 2 Datasets

PostPosted: Fri Dec 01, 2017 7:57 pm
by yiboj
This simple python script is used to subset and download L2/L2P datasets from the PODAAC server. The script in the previous post "Using Python to subset large gridded datasets (v2)" is mainly for L3/L4 dataset subsetting. The L2 subsetting script uses pydap package, and combines the PODAAC web granule spatial and temporal search service, and the OPeNDAP to subset and download most of the Level 2 and 2P datasets from PODAAC server.

Please note that you have to have the python software, pydap package, wget or curl installed on the computer.

You can download the script subset_dataset_l2.py here.

For example, to extract REMSS TMI data from 2010-01-01 to 2010-02-01 for a region bounded by -140 to -110 degrees longitude and -30 to 30 degrees latitude, this request can be made on the UNIX command line:

% python subset_dataset_l2.py -s 20100101 -f 20100201 -b -140 -110 -30 30 -x TMI-REMSS-L2P-v4

where TMI-REMSS-L2P-v4 is the short name for REMSS TMI dataset. Shortname is the required field for the script to run and the script will print out the help menu without the shortname. Shortname can be found from PODAAC web portal.

Re: Using Python to Subset and Download Level 2 Datasets

PostPosted: Sun Oct 06, 2019 4:55 pm
by jjakobos
Hello,

The link to subset_dataset_l2.py doesn't seem to be working? In any case, I am looking to subset L2 VIIRS data (id: PODAAC-GHVRS-2P061), which doesn't seem to be working with podaacpy l2ss. When I try to use the l2ss dataset_variables, it can't seem to find the dataset in the l2ss services directory.

Thank you!

Re: Using Python to Subset and Download Level 2 Datasets

PostPosted: Mon Oct 07, 2019 9:16 am
by yiboj
Hi,
Thanks for your inquiry.
We have updated the link which pointed to PODAAC Drive to download the script:
https://podaac-tools.jpl.nasa.gov/drive/files/common/sw/subsetting/subset_dataset_l2.py.

The GHRSST Level 2P OSPO dataset v2.61 dataset is in l2ss, and its ID is PODAAC-GHVRS-2PO61 not PODAAC-GHVRS-2P061, please check and let us know.

Regards,

PODAAC DE

Re: Using Python to Subset and Download Level 2 Datasets

PostPosted: Mon Oct 07, 2019 5:37 pm
by jjakobos
Awesome, thank you! I found the file, but have to debug (xml problem). In the meantime, I am running:
Code: Select all
import podaac.l2ss as l2
l = l2.L2SS()
result = l.granule_search(dataset_id='PODAAC-GHVRS-2PO61', bbox='0,0,180,90', start_time='2019-10-05T01:30:00Z',end_time='2019-10-07T00:00:00Z',start_index='1')
status=l.granule_download(result,'/data')

The search command seems to be working, but the download script seems to run until I get the following error:

Code: Select all
ConnectionError: HTTPSConnectionPool(host='podaac-tools.jpl.nasa.gov', port=443): Max retries exceeded with url: /l2ss-services/l2ss/subset/status?token=8e9a3eb0-df07-4514-afbf-434fd1a93ac5 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fdbd405a290>: Failed to establish a new connection: [Errno 110] Connection timed out',))

Any ideas on what might be going on? This is the first time I've tried to download subsetted L2 data from podaac, so I'm learning as I go...
Thanks again,
Julie

Edit: XML problem solved, need to use shortname not data_id.