METAMOD overview

Introduction

This document gives an overall description of METAMOD2. Installation of basic METAMOD2 software and deployment of new METAMOD2 applications are described in separate documents: Installation and Deployment.

The METAMOD2 software installed on a server can be used to run one or more METAMOD2 applications (or METAMOD2 instances). A METAMOD2 application is a web portal connected to a data base with metadata. The metadata will typically describe geoscience data files, and the web portal will offer a search interface that can display lists of files that meet user-supplied search criteria.

The central part of METAMOD2 is the data base containing metadata. This data base may be shared between several METAMOD2 applications. The metadata is obtained by extracting metadata from uploaded data files, harvesting metadata from other sources on the internet, entered using a web form or inserted manually in the XML archive (see below).

The METAMOD2 applications sharing a metadata data base is called a cluster. In its simplest form, a cluster comprise just one application, and this is often how METAMOD2 is used. However, by using several applications in a cluster, a major strength of METAMOD2 can be utilized: The ability to expose different parts of the metadata data base through different METAMOD2 applications. All metadata in the common data base are available for all the applications in a cluster. An application need not expose all metadata. Each application may be configured differently, so that only those metadata that are in agreement with the intended use of the application will be exposed through the web portal.

Data bases

The common metadata data base in a METAMOD2 cluster is divided into two parts:

XML archive

Metadata file archive based on XML. This file archive is subdivided into one directory tree for each METAMOD2 application in the cluster.

Metadatabase

SQL data base that is used as a fast search index for all metadata in the XML archive.

In addition to the common metadata data base, each METAMOD2 application has its own private user database:

User database

SQL Database for User administration.

Although METAMOD2 may be used merely as a search portal for metadata, where the data files described by the metatdata are situated elsewhere, METAMOD2 also facilitates handling and storing of actual data files. The data files managed in this way are stored in directories accessible from all the METAMOD2 applications in a cluster:

The data repository

A directory tree where METAMOD2 stores the data files uploaded to METAMOD2 through the upload module (see below). For the time being, only netCDF files are handled through the upload module, but the data repository may also comprise files with other formats.

One of the METAMOD applications in a cluster is responsible for the SQL metadata data base. Initialization and administration of the metadata database are done through this application, which is called the base application. Each METAMOD application must manage its own user database.

Although the METAMOD applications in a cluster may operate on different servers, they must share a common file system where the XML archive and the Data repository resides. Technically they may reside on different file systems, but for each file, the file paths on the various servers must be the same. Furthermore, for a smooth operation, the clocks on all the involved servers must be synchronized.

The SQL databases may reside on a separate server, different from any of the servers where the METAMOD instances run.

METAMOD functional modules

The METAMOD software comprise a broad range of functionality. The software is divided into different modules, each representing a separate set of related functionality. One such module is the BASE module, which is used in the base application described above. A METAMOD instance may use any combination of modules, except for the WEB module which is mandatory.

The only restriction is that one and only one application in a cluster must use the BASE module to initialize the metadata database. Each application must initialize its own user database however.

The METAMOD software comprise the following modules, which may be disabled unless otherwise noted:

WEB

Web interface for search, file upload, editing metadata, system administration and web services (RSS, OAI-PMH). This module is required, although some of the services may be disabled.

BASE

Used for configuring and creating the SQL metadata data base.

UPLOAD

Services for uploading (or registering) netCDF files (via web, ftp).

HARVEST

Service for harvesting metadata from internet servers offering OAI-PMH support.

PMH

OAI-PMH provider service. See http://www.openarchives.org/.

THREDDS

Used for automatic building of THREDDS catalogs that may be utilized by a THREDDS Data Server to access the data repository.

Directory structure

On each server where METAMOD should run, the generic METAMOD software must be available in some directory. If you installed METAMOD using a met.no Debian package it will be located in /opt/metno-metamod-2.x (where x denotes the minor version). In the description below, this directory is called the installation directory.

In addition to the installation directory you need a configuration directory. This directory contains configuration files, style sheets etc., and also files regulating metadata requirements. An example application configuration directory is included in the installation directory (the app/example subdirectory of the installation directory).

The main subdirectories of the installation directory are:

catalyst

Perl Catalyst framework files used for the WEB module

base

Files used in the BASE module, and files for initializing each user data base.

upload

Files only used in the UPLOAD module (scripts only, the user interface is implemented in the catalyst directory)

harvest

Files only used in the HARVEST module

thredds

Files only used in the THREDDS module

common

Files used by several modules

app

Contains the example application source directory, as well as default configuration settings

docs

METAMOD documentation directory