CLOSED - Metadata Compliance Checker UAT Testing Feedback

CLOSED - Metadata Compliance Checker UAT Testing Feedback

Postby podaac » Tue Feb 23, 2016 3:27 pm

PO.DAAC has recently deployed version 1.1 of the Metadata Compliance Checker (MCC) in the User Acceptance Tools (UAT) Server and would like to invite the users, to test this tool:

This Topic is no longer active. MCC Test Period ended on 1 March, 2017

https://podaac-uat.jpl.nasa.gov/mcc

The MCC is an online tool and web service designed to check and validate the contents of netCDF and HDF granules for the Climate and Forecast (CF) and Attribute Convention for Dataset Discovery (ACDD) metadata conventions. It produces quantitative scoring for these convention attributes as HTML, PDF or JSON outputs and works with both user locally uploaded data as well as data that are accessible via OPeNDAP. The MCC is based on open source software originally developed by NOAA that has been extended with additional functionality and reporting.

Your support in testing this new tool is greatly appreciated.
Please respond to this thread with your feedback.

MCC v 1.1 Test Period: 17 February, 2016 - 1 March, 2017

Thank you.
PO.DAAC Team
podaac
Site Admin
 
Posts: 456
Joined: Mon Oct 22, 2012 4:00 pm

Re: Metadata Compliance Checker (MCC) UAT Testing Feedback

Postby hyoklee » Tue Apr 26, 2016 8:30 am

I think OPeNDAP URL doesn't work with MCC.

For example, if I enter the following OPeNDAP URL:

http://test.opendap.org:80/opendap/data/nc/bears.nc

The response is:

There was a problem downloading your file. The server may be too busy or the file may not exist. URL: http://test.opendap.org/dap/data/nc/bears.nc

The MCC works with downloadable netCDF file, not OPeNDAP:

https://www.unidata.ucar.edu/software/n ... example.nc

Would you please fix it?
hyoklee
 
Posts: 2
Joined: Tue Apr 26, 2016 7:40 am

Re: Metadata Compliance Checker (MCC) UAT Testing Feedback

Postby podaac » Tue Apr 26, 2016 2:29 pm

Thank you for testing and for your feedback.

It looks like the OPENDAP URL in your example is not valid.

The example OPENDAP URL:
http://test.opendap.org:80/opendap/data/nc/bears.nc
Returns "access denied (403)” error.

Please try some other example or any of our OPENDAP datasets URLS:


i.e.: (Please see the screenshot below)
http://opendap.jpl.nasa.gov:80/opendap/ ... -fv01.0.nc

Please let us know if you have any questions or issues.

Thank you.
Attachments
Screen Shot 4_26_16 at 12.12.39 PM.png
Screen Shot 4_26_16 at 12.12.39 PM.png (74.92 KiB) Viewed 11786 times
podaac
Site Admin
 
Posts: 456
Joined: Mon Oct 22, 2012 4:00 pm

Re: Metadata Compliance Checker (MCC) UAT Testing Feedback

Postby hyoklee » Wed Apr 27, 2016 10:33 am

The example URL that you referred worked simply because your OPeNDAP server allowed direct access of the data source. By default, Hyrax (OPeNDAP server) doesn't allow direct access of the data source [1].

When your MCC web page says "OPeNDAP URL", I expected that MCC will utilize DAP2 protocol, not HTTP direct transfer. If MCC works with OPeNDAP URL, it should retrieve DDS/DAS and check conventions form there. This confused an avid OPeNDAP user like me.

Therefore, I'd like to suggest you to drop OPeNDAP from MCC [2]. Simply say "URL for Remote File" or "Remote File (enter URL)". MCC doesn't utilize OPeNDAP's DAP2 protocol at all based on my testing with HPD [3] so far. If you'd like to keep "OPeNDAP URL", please improve MCC to use OPeNDAP client such as PyDAP instead of transferring netCDF data directly from a simple web server such as GitHub.com that doesn't provide OPeNDAP service at all.

If you don't get what I mean, please feel free to contact me at hyoklee@hdfgroup.org.


[1] http://docs.opendap.org/index.php/Hyrax ... ptional.29
[2] http://podaac-uat.jpl.nasa.gov/mcc/
[3] https://wiki.earthdata.nasa.gov/display ... t+Designer
hyoklee
 
Posts: 2
Joined: Tue Apr 26, 2016 7:40 am

Re: Metadata Compliance Checker (MCC) UAT Testing Feedback

Postby mgangl » Wed Apr 27, 2016 1:38 pm

Hi Hyoklee,

You're correct. the MCC does not use the DAP protocol to check conventions, but rather downloads the data file via http and runs the analysis on the local file.

Internally, we will write a ticket and determine if and when we can add the DAP protocol to the MCC, and we will work on changing the name of the field to something more adequate (remote URL of file or something similar, like you suggested), although this update may take some time to go through our development process.

Thanks for your interest on the tool, we'll keep iterating and working on it to bring improvements that our users find worthwhile.
mgangl
 
Posts: 12
Joined: Wed Apr 27, 2016 1:31 pm

Re: Metadata Compliance Checker (MCC) UAT Testing Feedback

Postby mgangl » Tue Feb 07, 2017 11:16 am

MCC version 1.2

Hi all, we're happy to announce the release of MCC version 1.2 to our UAT environment. This newest release addresses a few issues while adding new functionality:

  • Added support for hierarchical groups to support more complex files (e.g. HDF files with nested groups). The tool's help page still says we don't support these, but we are working to update the documentation.
  • Fixed verbiage erroneously asking for OPeNDAP Links to remote files. We now ask for 'remote' files explicitly. This only works for HTTP/S products, not FTP or DAP protocol products. If your file is downloadable from an OPeNDAP instance, then that link should work.

As always, please use this forum to send suggestions, issues, and comments on the tool. We look forward to any and all feedback.

MCC v 1.2 Test Period: 1 February, 2017 - 1 March, 2017

Thank you.
PO.DAAC Team
mgangl
 
Posts: 12
Joined: Wed Apr 27, 2016 1:31 pm


Return to Metadata Compliance Checker (MCC)