PO.DAAC Data Management Best Practices

Introduction

This page contains the best data management practices at PO.DAAC. The content covers how PO.DAAC generally manages its data from acceptance, ingest, distribution, and retirement, along with the various documents and templates that are associated to those various stages. It also includes the recommended file formats (NetCDF, HDF) and metadata conventions (CF, ISO 19115, ACDD) for our data providers.

Dataset Acceptance

Every dataset that enters the PO.DAAC for archival and/or distribution must go through an acceptance process. The primary driver of this process is the Dataset Gap Analysis and Prioritization (DGAP). The DGAP provides the means for identification (See Figure 1 under Dataset Lifecycle) of individual datasets corresponding to a particular geophysical parameter, which are not yet being archived or distributed by the PO.DAAC. After identification, these datasets become ranked and prioritized according to their corresponding geophysical parameter to determine. The datasets of highest priority and ranking are then given the “green light” for the final phase of acceptance. In the final phase of acceptance, a system impact assessment is performed to assess the associated costs to the PO.DAAC to ingest the new datasets, and when approved by the Systems Engineer a Submission Agreement (a.k.a. Memorandum of Understanding) is signed between the dataset provider and the PO.DAAC manager. Once accepted, the new dataset is now ready to become integrated into the PO.DAAC.

Dataset Lifecycle

PO.DAAC’s Dataset Lifecycle provides us with a set of guidelines to govern the end-to-end archive and distribution of our data holdings. This framework provides a formalized process and requirements checklist useful for planning and managing datasets through the various phases of the Dataset Lifecycle. Its purpose is to ensure we meet the minimum standards and expectations for Quality, Quantity, Continuity, and Latency (QQCL), which are applied consistently for all archived datasets. Figure 1 depicts the flow of the dataset lifecycle as it transitions through various phases, beginning with identification.

Figure 1: A simple flow diagram of the Dataset Lifecycle through all phases. Solid red lines indicate chronological flow for all datasets while red dashed lines indicate conditions in which a dataset is either retired or replaced with a newer version. To the right of the flow diagram is a corresponding chronologically ordered table indicating the key deliverables throughout each Lifecycle phase. Check marks indicate which of the deliverables are required before the corresponding phase to the left is considered complete. DGAP = Dataset Gap Analysis and Prioritization. IS = Integrated Schedule. SIA = System Impact Assessment. IPA = Inter-Project Agreement. ICD = Interface Control Document. DMP = Data Management Plan. ORR = Operational Readiness Review. DOI = Digital Object Identifier. UG = User Guide. RP = Retirement Plan.

The Dataset Identification and Dataset Approval stages of the Dataset Lifecycle correspond directly with the information presented in the previous section on Dataset Acceptance. Dataset Integration begins with a complete harvesting and ingestion of metadata (both granule and collection-level) and concludes with full integration to all compatible access protocols (e.g., OPeNDAP, THREDDS), tools (e.g., HiTIDE, LAS, SOTO), and services (e.g., Web Portal, Web Services, EMS, GCMD, ECHO/REVERB). Once integrated, a review is conducted along with a formalized checklist to ensure all requirements and expectations are met, and the dataset becomes Fully Operational pending a successful review. Datasets that are Fully Operational come with the applicable levels of service to accommodate a service-oriented data distribution including user support, guide documentation, and accompanying read software. When new versions of an existing dataset become available, the Dataset Lifecycle process is repeated iteratively at the Dataset Approval stage. Dataset retirement is initiated on datasets which are deprecated by newer versions and removed from public visibility with reduced levels of service as determined by the dataset provider; aside from version deprecation, Dataset retirement may also result from a situation in which data may be quarantined due to a proliferation of erroneous data which contaminates the complete data record.

Related Links:
NASA Earth Science Data Preservation Content Specification:
https://cdn.earthdata.nasa.gov/conduit/upload/10607/NASA_ESD_Preservation_Spec.pdf

ESIP Preservation and Stewardship Cluster:
http://wiki.esipfed.org/index.php/Preservation_and_Stewardship

ESIP Data Study Cluster:
http://wiki.esipfed.org/index.php/Data_Study_Working_Group

ESIP Discovery Cluster:
http://wiki.esipfed.org/index.php/Discovery_Cluster

ESIP Documentation Cluster:
http://wiki.esipfed.org/index.php/Category:Documentation_Cluster

ESIP Information Quality Cluster:
http://wiki.esipfed.org/index.php/Information_Quality

Earth Science Data System Working Groups (ESDSWG):
https://earthdata.nasa.gov/esdswg

File Formats

PO.DAAC recommends that data providers implement their products using HDF5 or NetCDF4 file formats. These file formats contain versatile data models that supports a self describing, machine independent data format thus promoting interoperability, tool use and sharing of scientific data. Both data models have extensive applications in distributing NASA satellite data as well as used in many international satellite missions. Applications have included Level 2, 3 and 4 satellite data as well as in situ data.

Some of the features and support common to both formats are:

A versatile data model that can represent very complex data objects and a wide variety of metadata
A completely portable file format with no limit on the number or size of data objects in the collection
A software library that runs on a range of computational platforms, from laptops to massively parallel systems, and implements a high-level API with C, C++, Fortran 90, and Java interfaces
A rich set of integrated performance features that allow for access time and storage space optimizations
Community tools and applications for managing, manipulating, viewing, and analyzing the data in the collection

The HDF5 group

http://www.hdfgroup.org/HDF5/

Unidata and netCDF

http://www.unidata.ucar.edu/netcdf/

An overview of tools and services that work with netCDF

https://wiki.earthdata.nasa.gov/display/ESO/netCDF+Technical+Working+Group

Metadata Conventions

CF and ACDD Metadata Standards Overview for PO.DAAC Portal

In addition to advancing and providing recommendations on scientific file format standards as discussed above, PO.DAAC adopts, advocates for, and provides guidance on the usage of appropriate metadata standards for geospatial data. Metadata provide important descriptive information on data and file elements. Conformity to community developed and approved metadata standards facilitates the consistent and valid semantic interpretation of information and data critical for ensuring both efficient and automated data discovery and interoperability with tools and services across distributed and heterogeneous earth science data systems. While several established and emerging geospatial metadata frameworks exist (eg. FGDC, ISO19115), two metadata standards frameworks in particular are important in the remote sensing context, the “Attribute Conventions for Data Discovery” (ACDD) and the “Climate Forecast” (CF) convention. Implementation of ACDD and CF metadata by data providers is strongly encouraged, and the PO.DAAC works closely with providers to advise on metadata and file format aspects as part of our data archival acceptance policies. Here we provide an overview of these metadata frameworks with useful pointers to additional information. Data providers with further specific metadata questions who are considering data archival submissions should contact the PO.DAAC (email: podaac@podaac.jpl.nasa.gov).

CF
The Climate Forecast (CF) metadata convention, spearheaded by the Program for Model Diagnosis & Intercomparison (PCMDI) at Lawrence Livermore National Laboratory, provides a standards specification for both global file header and variable level file metadata attributes aimed at facilitating both discovery and interoperability of datasets used in climate science modeling, including remote sensing data. This includes standards for variable dimensioning, and variables holding both observational measurement data and georeference and time reference data linked to these plus associated auxiliary variables. The CF convention consists of a series of well-defined attributes, auxiliary variables, naming and value assignment conventions that provide a standard characterization of the contents of data files. This facilitates robust semantic interpretation and usage of data. Further information on CF metadata is available at http://cfconventions.org/. The current version of the CF standard is 1.7, and complete documentation of this latest specification is available at http://cfconventions.org/latest.html. The tables below summarize key global metadata attributes and variable level attributes associated with the CF convention recommended for usage with datasets slated for PO.DAAC archival.

ACDD
The ACDD convention, developed initially by UNIDATA as a complement to its netCDF self-describing scientific file format and now adopted by the ESIP Federation, provides a standards specification for global metadata attributes associated with the “header” portion of data files aimed at facilitating efficient and automated discovery of geospatial datasets. It consists of a series of standard attributes and naming conventions that provide a well-rounded and synoptic characterization of the scope and contents of data files, including for example the spatio-temporal extent of the file data, the type and processing level of the data, keyword descriptors, and details on the dataset provider. Complete documentation on ACDD metadata is available at http://wiki.esipfed.org/index.php/Category:Attribute_Conventions_Dataset_Discovery, and a listing of key ACDD global metadata and variable attributes recommended by the PO.DAAC minimally for inclusion in data files is given in the tables below.

Table 1. Key ACDD/CF Global Metadata Attributes defined and illustrated.

Attribute Name	Type	Definitions	Example Implementation	Priority	Source
title	string	A short phrase or sentence describing the dataset. In many discovery systems, the title will be displayed in the results list from a search, and therefore should be human readable and reasonable to display in a list of such names. This attribute is recommended by the NetCDF Users Guide (NUG) and the CF conventions.	title = "VIIRS L2P Sea Surface Skin Temperature" ;	required	ACDD 1.3, CF 1.7
summary	string	A paragraph describing the dataset, analogous to an abstract for a paper.	summary = "Sea surface temperature (SST) retrievals produced at the NASA OBPG for the Visible Infrared Imaging Radiometer Suite (VIIRS) sensor on the Suomi National Polar-Orbiting Partnership (Suomi NPP) platform. These have been reformatted to GHRSST GDS version 2 Level 2P specifications by the JPL PO.DAAC." ;	required	ACDD 1.3
keywords	string	A comma-separated list of key words and/or phrases. Keywords may be common words or phrases, terms from a controlled vocabulary (GCMD is often used), or URIs for terms from a controlled vocabulary (see also "keywords_vocabulary" attribute).	keywords = "Oceans, Ocean Temperature, Sea Surface Temperature , Sea Surface Skin Temperature" ;	required	ACDD 1.3
keywords_vocabulary	string	If you are using a controlled vocabulary for the words/phrases in your "keywords" attribute, this is the unique name or identifier of the vocabulary from which keywords are taken. If more than one keyword vocabulary is used, each may be presented with a prefix (e.g., "CF:NetCDF COARDS Climate and Forecast Standard Names") and a following comma, so that keywords may optionally be prefixed with the controlled vocabulary key.	keywords_vocabulary = "NASA Global Change Master Directory (GCMD) Science Keywords" ;	required	ACDD 1.3
Conventions	string	A comma-separated list of the conventions that are followed by the dataset. For files that follow this version of ACDD, include the string 'ACDD-1.3'. (This attribute is defined in NUG 1.7.)	Conventions = "CF-1.7, ACDD-1.3, ISO 8601";	required	ACDD 1.3, CF 1.7
id	string	An identifier for the data set, provided by and unique within its naming authority. The combination of the "naming authority" and the "id" should be globally unique, but the id can be globally unique by itself also. IDs can be URLs, URNs, DOIs, meaningful text strings, a local key, or any other unique string of characters. The id should not include white space characters.	id = "VIIRS_NPP-JPL-L2P-v2016.0" ;	recommended	ACDD 1.3
uuid	string	A uuid (Universal Unique Identifier) is a 128-bit number used to uniquely identify some object or entity on the Internet. Depending on the specific mechanisms used, a uuid is either guaranteed to be different or is, at least, extremely likely to be different from any other uuid generated until 3400 A.D.	uuid = "b6ac7651-7b02-44b0-942b-c5dc3c903eba" ;	required	PO.DAAC
naming_authority	string	The organization that provides the initial id (see above) for the dataset. The naming authority should be uniquely specified by this attribute. We recommend using reverse-DNS naming for the naming authority; URIs are also acceptable. Example: 'edu.ucar.unidata'.	naming_authority = "org.ghrsst" ;	recommended	ACDD 1.3
cdm_data_type	string	The data type, as derived from Unidata's Common Data Model Scientific Data types and understood by THREDDS. (This is a THREDDS "dataType", and is different from the CF NetCDF attribute 'featureType', which indicates a Discrete Sampling Geometry file in CF.)	cdm_data_type = "swath" ;	suggested	ACDD 1.3
history	string	Provides an audit trail for modifications to the original data. This attribute is also in the NetCDF Users Guide: 'This is a character array with a line for each invocation of a program that has modified the dataset. Well-behaved generic netCDF applications should append a line containing: date, time of day, user name, program name and command arguments.' To include a more complete description you can append a reference to an ISO Lineage entity; see NOAA EDM ISO Lineage guidance.	history = "VIIRS L2P created at JPL PO.DAAC by combining OBPG SNPP_SST and SNPP_SST3, and outputing to the GHRSST GDS2 netCDF file format" ;	required	ACDD 1.3, CF 1.7
source	string	The method of production of the original data. If it was model-generated, source should name the model and its version. If it is observational, source should characterize it. This attribute is defined in the CF Conventions. Examples: 'temperature from CTD #1234'; 'world model v.0.1'.	source = "VIIRS sea surface temperature observations for the OBPG" ;	required	ACDD 1.3, CF 1.7
platform	string	Name of the platform(s) that supported the sensor data used to create this data set or product. Platforms can be of any type, including satellite, ship, station, aircraft or other. Indicate controlled vocabulary used in platform_vocabulary.	platform = "Suomi-NPP" ;	recommended	ACDD 1.3
platform_vocabulary	string	Controlled vocabulary for the names used in the "platform" attribute.	platform_vocabulary = "GCMD platform keywords";	recommended	ACDD 1.3
instrument	string	Name of the contributing instrument(s) or sensor(s) used to create this data set or product. Indicate controlled vocabulary used in instrument_vocabulary.	sensor = "VIIRS" ;	recommended	ACDD 1.3
instrument_vocabulary	string	Controlled vocabulary for the names used in the "instrument" attribute.	instrument_vocabulary = "GCMD instrument keywords";	recommended	ACDD 1.3
processing_level	string	A textual description of the processing (or quality control) level of the data.	processing_level = "L2P" ;	required	ACDD 1.3
comment	string	Miscellaneous information about the data, not captured elsewhere. This attribute is defined in the CF Conventions.	comment = "L2P Core without DT analysis or other ancillary fields; Night, Start Node:Descending, End Node:Descending; WARNING Some applications are unable to properly handle signed byte values. If values are encountered > 127, please subtract 256 from this reported value; Quicklook" ;	required	ACDD 1.3, CF 1.7
standard_name_vocabulary	string	The name and version of the controlled vocabulary from which variable standard names are taken. (Values for any standard_name attribute must come from the CF Standard Names vocabulary for the data file or product to comply with CF.) Example: 'CF Standard Name Table v27'.	standard_name_vocabulary = "NetCDF Climate and Forecast (CF) Metadata Convention" ;	required	ACDD 1.3
acknowledgement	string	A place to acknowledge various types of support for the project that produced this data.	acknowledgment = "The VIIRS L2P sea surface temperature data are sponsored by NASA. Data may be freely distributed" ;	recommended	ACDD 1.3
license	string	Provide the URL to a standard or specific license, enter "Freely Distributed" or "None", or describe any restrictions to data access and distribution in free text.	license = "GHRSST and PO.DAAC protocol allow data use as free and open." ;	recommended	ACDD 1.3
metadata_link	string	A URL that gives the location of more complete metadata. A persistent URL is recommended for this attribute.	metadata_link = "http://podaac.jpl.nasa.gov/ws/metadata/dataset/?format=iso&shortName=VIIRS-JPL-L2P-v2016.0"	suggested	ACDD 1.3
product_version	string	Version identifier of the data file or product as assigned by the data creator. For example, a new algorithm or methodology could result in a new product_version.	product_version = "2016.0" ;	required	ACDD 1.3
references	string	Published or web-based references that describe the data or methods used to produce it. Recommend URIs (such as a URL or DOI) for papers or other references. This attribute is defined in the CF conventions.	references = "GHRSST Data Processing Specification v2r5" ;	suggested	ACDD 1.3, CF 1.7
creator_name	string	The name of the person (or other creator type specified by the creator_type attribute) principally responsible for creating this data.	creator_name = "JPL PO.DAAC" ;	required	ACDD 1.3
creator_email	string	The email address of the person (or other creator type specified by the creator_type attribute) principally responsible for creating this data.	creator_email = "ghrsst@jpl.nasa.gov" ;	required	ACDD 1.3
creator_url	string	The URL of the of the person (or other creator type specified by the creator_type attribute) principally responsible for creating this data.	creator_url = "http://podaac.jpl.nasa.gov" ;	required	ACDD 1.3
creator_type	string	Specifies type of creator with one of the following: 'person', 'group', 'institution', or 'position'. If this attribute is not specified, the creator is assumed to be a person.	creator_type = "Institution":	required	ACDD 1.3
creator_institution	string	The institution of the creator; should uniquely identify the creator's institution. This attribute's value should be specified even if it matches the value of publisher_institution, or if creator_type is institution.	creator_institution = = "JPL PO.DAAC/GHRSST";	required	ACDD 1.3
institution	string	The name of the institution principally responsible for originating this data. This attribute is recommended by the CF convention.	institution = "NASA Jet Propulsion Laboratory (JPL) Physical Oceanography Distributed Active Archive Center (PO.DAAC)/NASA Goddard Space Flight Center (GSFC), Ocean Biology Processing Group (OBPG)/University of Miami Rosential School of Marine and Atmospheric Science (RSMAS)" ;	required	ACDD 1.3, CF 1.7
project	string	The name of the project(s) principally responsible for originating this data. Multiple projects can be separated by commas, as described under Attribute Content Guidelines. Examples: 'PATMOS-X', 'Extended Continental Shelf Project'.	project = "Group for High Resolution Sea Surface Temperature" ;	required	ACDD 1.3
program	string	The overarching program(s) of which the dataset is a part. A program consists of a set (or portfolio) of related and possibly interdependent projects that meet an overarching objective. Examples: 'GHRSST', 'NOAA CDR', 'NASA EOS', 'JPSS', 'GOES-R'.	program= " NASA Earth Sciecne Data Information and System (ESDIS)"	required	ACDD 1.3
contributor_name	string	The name of any individuals, projects, or institutions that contributed to the creation of this data. May be presented as free text, or in a structured format compatible with conversion to ncML (e.g., insensitive to changes in whitespace, including end-of-line characters).	contributor_name = "PO.DAAC/OBPS/REMAS";	recommended	ACDD 1.3
contributor_role	string	The role of any individuals, projects, or institutions that contributed to the creation of this data. May be presented as free text, or in a structured format compatible with conversion to ncML (e.g., insensitive to changes in whitespace, including end-of-line characters). Multiple roles should be presented in the same order and number as the names in contributor_names.	contributor_role = "PO.DAAC convert the VIIRSS_NPP SST to GDS2 format, OBPS processed the L2P SST, and REMAS provided the algorithm model ";	recommended	ACDD 1.3
publisher_name	string	The name of the person (or other entity specified by the publisher_type attribute) responsible for publishing the data file or product to users, with its current metadata and format.	publisher_name = "The GHRSST Project Office" ;	required	ACDD 1.3
publisher_email	string	The email address of the person (or other entity specified by the publisher_type attribute) responsible for publishing the data file or product to users, with its current metadata and format.	publisher_email = "ghrsst-po@nceo.ac.uk" ;	required	ACDD 1.3
publisher_url	string	The URL of the person (or other entity specified by the publisher_type attribute) responsible for publishing the data file or product to users, with its current metadata and format.	publisher_url = "https://www.ghrsst.org" ;	required	ACDD 1.3
publisher_type	string	Specifies type of publisher with one of the following: 'person', 'group', 'institution', or 'position'. If this attribute is not specified, the publisher is assumed to be a person.	publisher_type = "instituion";	required	ACDD 1.3
publisher_institution	string	The institution that presented the data file or equivalent product to users; should uniquely identify the institution. If publisher_type is institution, this should have the same value as publisher_name.	publisher_institution = "PO.DAAC";	required	ACDD 1.3
geospatial_bounds	float	Describes the data's 2D or 3D geospatial extent in OGC's Well-Known Text (WKT) Geometry format (reference the OGC Simple Feature Access (SFA) specification). The meaning and order of values for each point's coordinates depends on the coordinate reference system (CRS). The ACDD default is 2D geometry in the EPSG:4326 coordinate reference system. The default may be overridden with geospatial_bounds_crs and geospatial_bounds_vertical_crs (see those attributes). EPSG:4326 coordinate values are latitude (decimal degrees_north) and longitude (decimal degrees_east), in that order. Longitude values in the default case are limited to the (-180, 180) range. Example: "POLYGON ((40.26 -111.29, 41.26 -111.29, 41.26 -110.29, 40.26 -110.29, 40.26 -111.29))".	geospatial_bounds = "(-143.09, -63.1404, -88.893, -36.7432)";	recommended	ACDD 1.3
geospatial_bounds_crs	string	The coordinate reference system (CRS) of the point coordinates in the geospatial_bounds attribute. This CRS may be 2-dimensional or 3-dimensional, but together with geospatial_bounds_vertical_crs, if that attribute is supplied, must match the dimensionality, order, and meaning of point coordinate values in the geospatial_bounds attribute. If geospatial_bounds_vertical_crs is also present then this attribute must only specify a 2D CRS. EPSG CRSs are strongly recommended. If this attribute is not specified, the CRS is assumed to be EPSG:4326. Examples: "EPSG:4979" (the 3D WGS84 CRS), "EPSG:4047".	geospatial_bounds_crs = "WGS84";	recommended	ACDD 1.3
geospatial_bounds_vertical_crs	string	The vertical coordinate reference system (CRS) for the Z axis of the point coordinates in the geospatial_bounds attribute. This attribute cannot be used if the CRS in geospatial_bounds_crs is 3-dimensional; to use this attribute, geospatial_bounds_crs must exist and specify a 2D CRS. EPSG CRSs are strongly recommended. There is no default for this attribute when not specified. Examples: "EPSG:5829" (instantaneous height above sea level), "EPSG:5831" (instantaneous depth below sea level), or "EPSG:5703" (NAVD88 height).	geospatial_bounds_vertical_crs = "EPSG:5831";	recommended	ACDD 1.3
geospatial_lat_min	float	Describes a simple lower latitude limit; may be part of a 2- or 3-dimensional bounding region. geospatial_lat_min specifies the southernmost latitude covered by the dataset.	geospatial_lat_min = -63.1404f ;	required	ACDD 1.3
geospatial_lat_max	float	Describes a simple upper latitude limit; may be part of a 2- or 3-dimensional bounding region. geospatial_lat_max specifies the northernmost latitude covered by the dataset.	geospatial_lat_max = -36.7432f ;	required	ACDD 1.3
geospatial_lat_units	string	Units for the latitude axis described in "geospatial_lat_min" and "geospatial_lat_max" attributes. These are presumed to be "degree_north"; other options from udunits may be specified instead.	geospatial_lat_units = "degrees_north" ;	required	ACDD 1.3
geospatial_lat_resolution	float	Information about the targeted spacing of points in latitude. Recommend describing resolution as a number value followed by the units. Examples: '100 meters', '0.1 degree'. For level 1 and 2 swath data this is an approximation of the pixel resolution.	geospatial_lat_resolution = 0.0075f ;	required	ACDD 1.3
geospatial_lon_min	float	Describes a simple longitude limit; may be part of a 2- or 3-dimensional bounding region. geospatial_lon_min specifies the westernmost longitude covered by the dataset. See also geospatial_lon_max.	geospatial_lon_min = -143.09f ;	required	ACDD 1.3
geospatial_lon_max	float	Describes a simple longitude limit; may be part of a 2- or 3-dimensional bounding region. geospatial_lon_max specifies the easternmost longitude covered by the dataset. Cases where geospatial_lon_min is greater than geospatial_lon_max indicate the bounding box extends from geospatial_lon_max, through the longitude range discontinuity meridian (either the antimeridian for -180:180 values, or Prime Meridian for 0:360 values), to geospatial_lon_min; for example, geospatial_lon_min=170 and geospatial_lon_max=-175 incorporates 15 degrees of longitude (ranges 170 to 180 and -180 to -175).	geospatial_lon_max = -88.893f ;	required	ACDD 1.3
geospatial_lon_units	string	Units for the longitude axis described in "geospatial_lon_min" and "geospatial_lon_max" attributes. These are presumed to be "degree_east"; other options from udunits may be specified instead.	geospatial_lon_units = "degrees_east" ;	required	ACDD 1.3
geospatial_lon_resolution	float	Information about the targeted spacing of points in longitude. Recommend describing resolution as a number value followed by units. Examples: '100 meters', '0.1 degree'. For level 1 and 2 swath data this is an approximation of the pixel resolution.	geospatial_lon_resolution = 0.0075f ;	required	ACDD 1.3
geospatial_vertical_min	float	Describes the numerically smaller vertical limit; may be part of a 2- or 3-dimensional bounding region. See geospatial_vertical_positive and geospatial_vertical_units.	geospatial_vertical_min = 0.00f;	recommended	ACDD 1.3
geospatial_vertical_max	float	Describes the numerically larger vertical limit; may be part of a 2- or 3-dimensional bounding region. See geospatial_vertical_positive and geospatial_vertical_units.	geospatial_vertical_max = 1000.00f;	recommended	ACDD 1.3
geospatial_vertical_resolution	float	Information about the targeted vertical spacing of points. Example: '25 meters'	geospatial_vertical_resolution = 25.0f;	suggested	ACDD 1.3
geospatial_vertical_units	string	Units for the vertical axis described in "geospatial_vertical_min" and "geospatial_vertical_max" attributes. The default is EPSG:4979 (height above the ellipsoid, in meters); other vertical coordinate reference systems may be specified. Note that the common oceanographic practice of using pressure for a vertical coordinate, while not strictly a depth, can be specified using the unit bar. Examples: 'EPSG:5829' (instantaneous height above sea level), 'EPSG:5831' (instantaneous depth below sea level).	geospatial_vertical_units = 'meters';	suggested	ACDD 1.3
geospatial_vertical_positive	string	One of 'up' or 'down'. If up, vertical values are interpreted as 'altitude', with negative values corresponding to below the reference datum (e.g., under water). If down, vertical values are interpreted as 'depth', positive values correspond to below the reference datum. Note that if geospatial_vertical_positive is down ('depth' orientation), the geospatial_vertical_min attribute specifies the data's vertical location furthest from the earth's center, and the geospatial_vertical_max attribute specifies the location closest to the earth's center.	geospatial_vertical_positive = 'down';	suggested	ACDD 1.3
time_coverage_start	string	Describes the time of the first data point in the data set. Use the ISO 8601:2004 date format, preferably the extended format as recommended in the Attributes Content Guidance section.	time_coverage_start = "2016-09-01T08:12:01" ;	required	ACDD 1.3
time_coverage_end	string	Describes the time of the last data point in the data set. Use ISO 8601:2004 date format, preferably the extended format as recommended in the Attributes Content Guidance section.	time_coverage_end = "2016-09-01T08:17:59" ;	required	ACDD 1.3
time_coverage_duration	string	Describes the duration of the data set. Use ISO 8601:2004 duration format, preferably the extended format as recommended in the Attributes Content Guidance section.	time_coverage_duration = "P4Y6M15DT20H30M40S";	recommended	ACDD 1.3
time_coverage_resolution	string	Describes the targeted time period between each value in the data set. Use ISO 8601:2004 duration format, preferably the extended format as recommended in the Attributes Content Guidance section.	time_coverage_resolution = "00:05:58";	recommended	ACDD 1.3
date_created	string	The date on which this version of the data was created. (Modification of values implies a new version, hence this would be assigned the date of the most recent values modification.) Metadata changes are not considered when assigning the date_created. The ISO 8601:2004 extended date format is recommended, as described in the Attribute Content Guidance section.	date_created = "2016-10-14T21:00:25" ;	required	ACDD 1.3
date_modified	string	The date on which the data was last modified. Note that this applies just to the data, not the metadata. The ISO 8601:2004 extended date format is recommended, as described in the Attributes Content Guidance section.	date_modified = "2016-10-14T21:00:25" ;	recommended	ACDD 1.3
date_issued	string	The date on which this data (including all modifications) was formally issued (i.e., made available to a wider audience). Note that these apply just to the data, not the metadata. The ISO 8601:2004 extended date format is recommended, as described in the Attributes Content Guidance section.	date_issued = "2016-10-14T21:00:25" ;	recommended	ACDD 1.3
date_metadata_modified	string	The date on which the metadata was last modified. The ISO 8601:2004 extended date format is recommended, as described in the Attributes Content Guidance section.	date_metadata_modified = "2016-10-14T21:00:25" ;	recommended	ACDD 1.3

Table 2. ACDD/CF Measurement /Auxiliary/Quality Variable Attributes.

Both variable dimensions and the associated attribute list are included as part of the variable declaration in netCDF implementations, and take the general form:

TYPE VariableName (DimensionX, ..);                                   eg. float sss(IdLat, IdLon);
                :Attribute1 :long_name =
                …                                                                               …
                :AttributeN                                                                   :comment =

Attribute Name	Type	Definitions	Example Implementation	Priority	Source	Variable Type
long_name	string	A long descriptive name for the variable (not necessarily from a controlled vocabulary). This attribute is recommended by the NetCDF Users Guide, the COARDS convention, and the CF convention.	long_name = "sea surface temperature" ;	required	ACDD 1.3, CF 1.7	measurement/ auxillary/quality variables
standard_name	string	A long descriptive name for the variable taken from a controlled vocabulary of variable names. We recommend using the CF convention and the variable names from the CF standard name table (http://cfconventions.org/Data/cf-standard-names/36/build/cf-standard-nam...). This attribute is recommended by the CF convention.	standard_name = "sea_surface_skin_temperature" ;	required	ACDD 1.3, CF 1.7	measurement/ auxillary/quality variables
units	string	The units of the variable's data values. This attribute value should be a valid udunits string. The "units" attribute is recommended by the NetCDF Users Guide, the COARDS convention, and the CF convention (http://www.unidata.ucar.edu/software/udunits/udunits-1/udunits.txt).	units = "kelvin" ;	required	ACDD 1.3, CF 1.7	measurement/ auxillary/quality variables
coverage_content_type	string	An ISO 19115-1 code to indicate the source of the data --MD_CoverageContentTypeCode (https://geo-ide.noaa.gov/wiki/index.php?title=ISO_19115_and_19115-2_Code...). For example, image, thematicClassification, physicalMeasurement, auxiliaryInformation, qualityInformation, referenceInformation, modelResult, or coordinate.	coverage_content_type = "physicalMeasurement" ;	required	ACDD 1.3	measurement/ auxillary/quality variables
valid_range	variable type	Comma separated minimum and maximum values of the physical quantity defining the valid measurement range.	valid_range = 0.0f, 500.0f;	required	CF 1.7	measurement/ auxillary/quality variables
coordinates	string	This attribute contains a space separated list of all the coordinates corresponding to the variable. The list should contain all the auxiliary coordinate variables and optionally the coordinate variables	coordinates = "lat lon";	required	CF 1.7	measurement/ auxillary/quality variables
scale_factor	variable type	Slope of scaling relationship applied to transform measurement data to appropriate geophysical quantity representations. Should not be used if the scale_factor is '1' and add_offset is '0'	scale_factor = 0.005f ;	required	CF 1.7	measurement/ auxillary/quality variables
add_offset	variable type	Intercept of scaling relationship applied to transform measurement data to appropriate geophysical quantity representations. Should not be used if the scale_factor is '1' and add_offset is '0'	add_offset = 273.15f ;	required	CF 1.7	measurement/ auxillary/quality variables
_FillValue	variable type	Assigned value in the data file designating a null or missing observation	_FillValue = -32767s ;	required	CF 1.7	measurement/ auxillary/quality variables
grid_mapping	string	Describes the horizontal coordinate system used by the data. The grid_mapping attribute should point to a variable which would contain the parameters corresponding to the coordinate system. There are typically several parameters associated with each coordinate system. CF defines a separate attributes for each of the parameters. Some examples are "semi_major_axis", "inverse_flattening", "false_easting"	grid_mapping = "semi_major_axis";	recommended	CF 1.7	measurement/ auxillary/quality variables
comment	string	Optional attribute field allowing provision of further free-form information about the variable	comment = "sea surface temperature from 11 and 12 um (thermal IR) channels" ;	required	CF 1.7	measurement/ auxillary/quality variables
flag_masks	variable type	A number of independent Boolean (binary) conditions using bit field notation and setting unique bits whose values are associated with a list of descriptive phrases in attribute flag_meanings. The flag_masks attribute is the same type as the variable to which it is attached, and contains a list of values matching unique bit fields. (See CF document section 3.5; http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/cf-conventions.html#flags) In this example variableY has 6 unique flag_masks bit flag states which are defined by the strings in flag_meanings variableY:flag_masks = 1b, 2b, 4b, 8b, 16b, 32b ; variableY:flag_meanings = "land_contamination open_ocean coastal_ocean lake river ice_contamination " ;	flag_masks = 1b, 2b, 4b, 8b, 16b;	required	CF 1.7	quality variable
flag_meanings	string	A list of strings that defines the the physical meaning of each flag_masks bit field or flag_values scaler field with a single text string. The strings are often phrases with words catenated with underscores, and strings are separated by a single space. CF allows a single variable to contain both flag_values and flag_masks. The interpretation of the flags in such cases is slightly tricky. In such cases flag_masks is used to "group" a set of flag_values into a nested conditional. Please see the example 3.5 in the CF document on how to interpret flag_meanings in such cases. It is recommended that Boolean (flag_masks) and enumerated flags (flag_values) be kept in separate variables.	flag_meanings = "microwave land ice lake river" ;	required	CF 1.7	quality variable
flag_values	variable type	flag_values consists of an enumerated list of status flags indicating unique conditions whose meaning is described by the commensurate list of descriptive phrases in attribute flag_meanings. The status flags are scaler of the same type as the variable. In this example, data points in variableX have one of 3 possible status flags having values of 0, 1, or 2 variableX:flag_values = 0b, 1b, 2b ; variableX:flag_meanings = "good_quality suspect_data sensor_nonfunctional”	flag_values = 1b, 5b ;	required	CF 1.7	quality variable

Table 3. Georeferencing Coordinate Variable Attributes.

Both variable dimensions and the associated attribute list are included as part of the variable declaration in netCDF implementations, and take the general form:

TYPE VariableName (DimensionX);                                        eg. float Lat(IdLat);
                :Attribute1 :long_name =
                … …
                :AttributeN :units =

One can see that many attributes are shared across Measurement and Georeferencing variables.

Attribute Name	Type	Definitions	Example Implementation	Priority	Source	Variable Type
long_name	string	Custom/long descriptive name of variable	long_name = "latitude" or "longitude";	required	ACDD 1.3, CF 1.7	geo-reference variables
standard_name	string	Standard variable name used to describe the georeferencing variable (eg. latitude, longitude, height)	standard_name = "latitude" or "longitude";	required	ACDD 1.3, CF 1.7	geo-reference variables
units	string	Standard unit name for the standard georeferencing variable (e.g., "degrees_north", "degrees_east", "m"	units = "degrees_north" or "degree_east";	required	ACDD 1.3, CF 1.7	geo-reference variables
axis	string	Corresponding variable axis for plotting (e.g., X, Y, Z)	axis = "X" or "Y";	required	CF 1.7	geo-reference variables
_FillValue	float	Assigned value in the data file designating a null or missing observation. NASA best practices specifies that for satellite datasets there should not be a _FillVlalue for these geolocation variables.	_FillValue = -9999.0f ;	required	CF 1.7	geo-reference variables
valid_min	float	The minimum values of georeferencing variables (eg. latitude, longitude, height)	valid_min = -90.f (for lat); valid_min = -180.f (for lon);	required	CF 1.7	geo-reference variables
valid_max	float	The maximum values of georeferencing variables (eg. latitude, longitude, height)	valid_max = 90.f (for lat); valid_max = 180.f (for lon);	required	CF 1.7	geo-reference variables
comment	string	Optional attribute field allowing provision of further free-form information about the variable	comment = "geographical coordinates, WGS84 projection" ;	required	CF 1.7	geo-reference variables

Table 4. Temporal Coordinate Variable Attributes.

Both variable dimensions and the associated attribute list are included as part of the variable declaration in netCDF implementations, and take the general form:

TYPE VariableName (DimensionT);                                         eg. float MeasurementTimes(t);
                :Attribute1 :long_name =
                … …
                :AttributeN :_FillValue =

Here too many attributes are shared with Measurement and Georeferencing variables, but with a specialized usage specification of the 'units' attribute as shown below.

Attribute Name	Type	Definitions	Example Implementation	Priority	Source	Variable Type
long_name	string	Custom/long descriptive name of variable	long_name = "time of measurement" ;	required	ACDD 1.3, CF 1.7	time variables
standard_name	string	Standard variable name used to describe the temporal variable (ie. time)	standard_name = “time”;	required	ACDD 1.3, CF 1.7	time variables
units	string	Standard unit descriptor (e.g., days, hours, seconds etc) cited against a standard reference date ("since".. date/time in ISO 8601 format)	units = "seconds since 1981-01-01 00:00:00" ;	required	ACDD 1.3, CF 1.7	time variables
axis	string	Corresponding variable axis for plotting (e.g., T)	axis = "T";	required	CF 1.7	time variables
_FillValue	variable type	Assigned value in the data file designating a null or missing observation. NASA best practices specifies that for satellite datasets there should not be a _FillVlalue for time variables.	_FillValue = -9999.0f;	required	CF 1.7	time variables
comment	string	Optional attribute field allowing provision of further free-form information about the variable	comment = "time of first sensor observation" ;	required	CF 1.7	time variables

In addition to conventions for Flag related attributes, the CF1.7 convention now also provides specifications for auxiliary variables that further qualify attributes of the core variable types listed above. Examples include:

the grid_mapping attribute and variables that describe the mapping projection between coordinate variables and the true latitude and longitude coordinates.
Grid cell-related attributes and variables such as bounds and cell_methods that qualify the gridding regime used and aggregate operations conducted to yield grid data values are now available
Standard attribute structures for the representation of complex sampling geometries, from point data to time series to profiles to trajectories.

Full descriptions of these capabilities and further information on the core attribute types summarized above are provided in the CF documentation available at http://cfconventions.org/.

ISO 19115

ISO 19115 is a type of metadata that facilitates machine readability. At PO.DAAC we have consolidated web services (http://podaac.jpl.nasa.gov/ws) that will convert metadata to make it ISO 19115 compliant.

PO.DAAC

Navigation

You are here

PO.DAAC Data Management Best Practices

In the sections below you will find:

Dataset Acceptance

Dataset Lifecycle

File Formats

Metadata Conventions