Using Python to Subset and Download Level 2 Datasets

Using Python to Subset and Download Level 2 Datasets

Postby yiboj » Fri Dec 01, 2017 7:57 pm

This simple python script is used to subset and download L2/L2P datasets from the PODAAC server. The script in the previous post "Using Python to subset large gridded datasets (v2)" is mainly for L3/L4 dataset subsetting. The L2 subsetting script uses pydap package, and combines the PODAAC web granule spatial and temporal search service, and the OPeNDAP to subset and download most of the Level 2 and 2P datasets from PODAAC server.

Please note that you have to have the python software, pydap package, wget or curl installed on the computer.

You can download the script subset_dataset_l2.py here.

For example, to extract REMSS TMI data from 2010-01-01 to 2010-02-01 for a region bounded by -140 to -110 degrees longitude and -30 to 30 degrees latitude, this request can be made on the UNIX command line:

% python subset_dataset_l2.py -s 20100101 -f 20100201 -b -140 -110 -30 30 -x TMI-REMSS-L2P-v4

where TMI-REMSS-L2P-v4 is the short name for REMSS TMI dataset. Shortname is the required field for the script to run and the script will print out the help menu without the shortname. Shortname can be found from PODAAC web portal.
yiboj
 
Posts: 93
Joined: Mon Mar 30, 2015 11:22 am

Re: Using Python to Subset and Download Level 2 Datasets

Postby jjakobos » Sun Oct 06, 2019 4:55 pm

Hello,

The link to subset_dataset_l2.py doesn't seem to be working? In any case, I am looking to subset L2 VIIRS data (id: PODAAC-GHVRS-2P061), which doesn't seem to be working with podaacpy l2ss. When I try to use the l2ss dataset_variables, it can't seem to find the dataset in the l2ss services directory.

Thank you!
jjakobos
 
Posts: 2
Joined: Thu Oct 03, 2019 2:29 pm

Re: Using Python to Subset and Download Level 2 Datasets

Postby yiboj » Mon Oct 07, 2019 9:16 am

Hi,
Thanks for your inquiry.
We have updated the link which pointed to PODAAC Drive to download the script:
https://podaac-tools.jpl.nasa.gov/drive/files/common/sw/subsetting/subset_dataset_l2.py.

The GHRSST Level 2P OSPO dataset v2.61 dataset is in l2ss, and its ID is PODAAC-GHVRS-2PO61 not PODAAC-GHVRS-2P061, please check and let us know.

Regards,

PODAAC DE
yiboj
 
Posts: 93
Joined: Mon Mar 30, 2015 11:22 am

Re: Using Python to Subset and Download Level 2 Datasets

Postby jjakobos » Mon Oct 07, 2019 5:37 pm

Awesome, thank you! I found the file, but have to debug (xml problem). In the meantime, I am running:
Code: Select all
import podaac.l2ss as l2
l = l2.L2SS()
result = l.granule_search(dataset_id='PODAAC-GHVRS-2PO61', bbox='0,0,180,90', start_time='2019-10-05T01:30:00Z',end_time='2019-10-07T00:00:00Z',start_index='1')
status=l.granule_download(result,'/data')

The search command seems to be working, but the download script seems to run until I get the following error:

Code: Select all
ConnectionError: HTTPSConnectionPool(host='podaac-tools.jpl.nasa.gov', port=443): Max retries exceeded with url: /l2ss-services/l2ss/subset/status?token=8e9a3eb0-df07-4514-afbf-434fd1a93ac5 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fdbd405a290>: Failed to establish a new connection: [Errno 110] Connection timed out',))

Any ideas on what might be going on? This is the first time I've tried to download subsetted L2 data from podaac, so I'm learning as I go...
Thanks again,
Julie

Edit: XML problem solved, need to use shortname not data_id.
jjakobos
 
Posts: 2
Joined: Thu Oct 03, 2019 2:29 pm


Return to Data Recipes