Download Multiple Data Files from PO.DAAC Drive Using curl

Download Multiple Data Files from PO.DAAC Drive Using curl

Postby yiboj » Thu Sep 20, 2018 8:54 am

PO.DAAC Drive can be accessed both by wget and curl command on linux system. The recipe "Download Multiple Data Files from PO.DAAC Drive Using wget" shows how to use wget command, and this recipe will focus on the curl command.

curl is a tool to transfer data from or to a server, using one of the
supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP,
TELNET and TFTP). The command is designed to work without user interaction.

curl offers a busload of useful tricks like proxy support, user authentication, FTP upload, HTTP post, SSL connections, cookies, file transfer resume, Metalink, and more. As you will see below, the number of features will make your head spin!

The major difference between wget and curl is that wget can download files recursively and curl can upload file to the server.

The curl command allows you to download as well as upload data through the command line in Linux. Following is its syntax:
Code: Select all
curl [options] [URL...]

1. curl Command Options

Here is the list of a few key options frequently used:

urlOne or multiple URLs that will be fetched in sequence..
-u user:passwordThe username and password to use for server authentication.
-o filenameWrite output to filename instead of stdout.
-OWrite output to a local file named like the remote file we get. (Only the file part of the remote file is used, the path is cut off.)
--compressedRequest a compressed response using one of the algorithms curl supports (gzip), and save the uncompressed document.

2. Download multiple files from PO.DAAC Drive

In order to access PO.DAAC Drive, all users are required to be registered with NASA Earthdata Login. User can login to the PO.DAAC Drive using the following link Figure 1 shows the WebDAV/Programmatic API credentials which will be used later to access the files through wget command. Please note that the password is encrypted, it is different from the Earthdata Login password.
Figure 1: PO.DAAC Drive Login
podaac_drive_login.png (32.3 KiB) Viewed 4533 times

We take the GHRSST SST Level 2 AMSRE datset from REMSS as an example (Figure 2).
Figure 2: PO.DAAC AMSRE v7 Dataset
podaac_amsre_access.png (116.92 KiB) Viewed 4533 times

* To download one data file
Code: Select all
Posts: 130
Joined: Mon Mar 30, 2015 11:22 am

Return to PO.DAAC Drive