Metadata Management User Guide

Your data is streaming through GSN into SensorMap and you have users at several sites observing this data and using it for different purposes. If your data is public, you may not know who your users are.......You go into the field, repair a broken sensor, replace another, and notice that a third sensor is frozen and therefore not working.....How do you communicate this to the users? If you merely write it in a notebook, they will never know about it.

The SwissEx metadata management wiki extensions are designed to aid projects by creating a metadata database about the hardware used in their experimental fieldwork as well as about the site itself. Data is entered using a user interface designed to guide the user through the process of entering hardware information and observations. The semantic wiki then automatically stores this metadata in an SQL database so that it may be queried along with the corresponding data.

The database created using the SwissEx extensions may be read using SPARQL queries to provide information entered through the wiki interface directly where it is needed: alongside the data.

Contents 1 Metadata Management User Guide 1.1 Fieldsites 1.1.1 Entering a fieldsite 1.1.2 How it appears in the fieldsite database 1.2 Measurement Locations 1.3 Sensor models 1.4 Sensor Instances 1.5 Transfer Function 1.6 Experimental Method 1.7 Measurements 1.8 Observations 1.9 Invalid Data 1.10 Editing a page 1.11 Deleting information

Fieldsites

The main entry point for the metadata is through the fieldsites database: http://www.swiss-experiment.ch/index.php/Fieldsite:Home (accessible through the 'Data Access Portal' link on the front page). This page provides a map/list of all fieldsites entered using the forms provided.

Entering a fieldsite

Once the 'Add a fieldsite' link on the fieldsite database page has been clicked, the user is presented with two options: a box where the namespace (always Fieldsite:) and pagename can be inserted; or a link to click that creates a random page name. Once one of these options has been completed, the user will be presented with a form into which the relevant information should be added before clicking 'save page'. When typing inside the fields within the form, previously entered text/values will automatically be shown where appropriate.

The following fields are provided in the form:

• Deployment: The name of your fieldsite
• Description: This should consist of searchable keywords describing what is measured at the site.
• Coordinates: Enter the coordinates of the fieldsite into the box below the map, press 'look up' and the coordinates will be inserted into the box above the map in the correct format. Alternatively, click on the map to select your coordinates.
• Altitude: The altitude of the fieldsite in meters
• Swiss Coordinates: These may be of use if the semantic map point extension is used
• Start/End date: The dates from and until which the fieldsite was used to collect data
• Institute responsible: The name of your institute so that people may get in contact if they are interested in the site. This is intentionaly left without a specific contact person. If you are in a large organisation, you may wish to specify your department.
• Image link: follow the link provided to upload a photograph of your fieldsite (this link can be found at the bottom of every page when logged in), then insert the last part of the URL (Image:filename.jpg) into the box, enclosed by double square brackets [[Image:filename.jpg]]
• Free text: please use this section to describe the fieldsite in more detail, including exactly what is/has been measured there.

After clicking save, the user will be presented with a table containing all of the information added plus a map of the location and another map/table which will contain the measurement positions once they have been entered.

How it appears in the fieldsite database

Once the fieldsite page has been produced, it will automatically be shown in the table and map on the fieldsite database page (these are simply formed by queries to the database). The fieldsite page can be extended to provide any additional information that you find helpful (not in database form). To do this, simply click on the edit link at the bottom of the page. The information created by the forms is shown, enclosed in curved brackets{}. This information should not be disturbed, but you can add additional information using wiki syntax above or below the block of information enclosed by the brackets.

Measurement Locations

The measurement locations are positions, such as fixed stations or revisited sites where measurements are taken within the fieldsite.

Within a fieldsite page, a link is provided to add measurement locations. The linked form automatically generates a random page name.* Enter your measurement location information (note that Swiss Coordinates are not mandatory but will allow you to use geo-referenced images) and click on 'save page'. You will be presented with the measurement location page and a link will be automatically placed in the measurement locations table and map in the fieldsite page.

The measurement locations page shows the results of two queries. One query shows the sensors and database parameters corresponding to each sensor deployment at that location (incl. dates), the other query shows the observations corresponding to this location.

THE MINIMAL USER CAN STOP HERE, JUST HAVING A FIELDSITE PAGE AND MEASUREMENT LOCATION PAGES. OBSERVATIONS CAN BE ADDED TO THE MEASUREMENT LOCATION PAGE USING THE OBSERVATIONS SECTION BELOW. IF THE FIELDSITE CONTAINS MANY LONG TERM SENSORS, THE USER SHOULD READ ON.

* The page name can be changed to something more meaningful to users only after the page has been created. At the bottom of the saved measurement location page, click Move, enter the new name into the linked form (do not change the Fieldsite: part), and click Move page. For this name change to show up on the main fieldsite page, return to the measurement location page, click Edit information, and click save. Check that the location now appears with the correct name on your fieldsite page.

Sensor models

Sensor types can be identified according to the model number and manufacturer. This information is stored along with specific information about the model in the sensor model page.

A sensor model can be registered by clicking on the link on the measurement location page.

Specific information that is useful for describing the sensor model includes:

• Lower range: the lower bound of measurable values from the sensor
• Upper range: the upper bound of measurable values from the sensor
• NaN value: this information can be used by GSN to convert specified values to empty cells in the database, hence providing gaps in the data instead of unknown values
• Measurement error: the accuracy to which the value can be measured
• Decimal places: the number of decimal places provided by the instrument

Once 'save page' is pressed, the following page is shown. The table shows the results of a query for all instances of this sensor model registered at various fieldsites.

Sensor Instances

A sensor instance is a specific sensor which can be identified by its serial number.

A sensor instance can be registered into the system using the link on the measurement location page.

After clicking the link, the user is presented with a form containing two parameters (Sensor Model and serial number):

Once completed and saved, the system returns a page with a number of query results:

• Sensor model: the link to the sensor model and its parameters
• Transfer function: transfer functions relating to this sensor instance
• Database parameters: database parameters fed from this sensor
• Sensor deployments: deployments of this sensor instance within the fieldsite (there may be more than one entry, each corresponding to a database parameter; a database parameter may be related to multiple sensor instances over time)

Transfer Function

A transfer function relates to a function combining all types of calibration which can be applied to the sensor in order to interpret the measured values into a true parameter value.

A transfer function can be registered using the link on the sensor instance page.

Multiple transfer functions can be recorded for each sensor instance, along with a corresponding date. It is assumed that a transfer function is valid for a sensor from the date recorded on the form until the next available date corresponding to the new transfer function.

The form has simply 3 fields: serial number, transfer function and date.

Once this form is saved, the transfer function page is shown, which simply records the function.

Experimental Method

The experimental method allows you to record exactly how the measurement was taken. For example, if you have used a handheld instrument: how you used the instrument, how you inserted the sensor, etc. (i.e. everything that is required to understand a non-standardised measurement).

The experimental method is entered using the following form (accessed via the database parameter page):

The fields required on the form are as follows:

• Database table name: as for the database parameter table, i.e. the virtual sensor name for GSN or a file link for an excel file
• Database parameter name: the parameter within the virtual sensor/spreadsheet
• Experimental method name: a name to distinguish this method in the list of experimental methods
• URL reference: in case the experimental method exists somewhere on the web
• Literature reference: a reference if the experimental method is described in published literature
• Method property: this is a link to a page within the wiki where you can describe your experimental method, e.g. Wannengrat:Home (the last part of the wiki page URL)

Once 'save page' is pressed, the page will look as follows, with a query showing all available experimental methods.

Measurements

The measurement pages are the 'glue' that holds the system together.

Once you have created your sensor models and their various sensor instances and have set up your database parameters, you are ready to go into the field and deploy your sensors (or manually measure data). When you deploy a sensor info.pnga sensor can also be something as simple as a ruler to a measurement location, note down the sensor's serial number and put it into this page. When you go to replace that sensor at a later date, you can fill in the end date of your sensor deployment and fill out another form to deploy a different sensor at that point. Both of these sensors will be deployed to the same location and will feed the same database parameter, so when you open the database parameter, you will see which sensors were used between which times, and therefore if and when transfer functions should be applied to the data.

Using a different approach, if you are replacing sensors and their associated loggers/nodes, you will find not only which sensors were measuring at a specific measurement location between which times, but also which logger this was logged through. Therefore, you can join the data streams from two existing logger virtual sensors to create a single irtual sensor corresponding to a measurement location.

By clicking the deployment link on the measurement location page, the user is presented with the following form:

This form contains a number of fields:

• Serial No.: Unique identifier of the sensor.
• Measurement location: The measurement location to which the sensor/logger is deployed.
• Relative X,Y,Z coordinates: The coordinates in relation to the logger. If on a flat plane, these are +ve X=East, +ve Y= North, +ve Z=up. This coordinate system should always be referenced to the logger and Z should always be height/depth.
• Local slope angle: The slope in the area immediately around the measurement location.
• From/to date: The dates between which the sensor/logger is deployed at this position.
• Image: Any relevant images of the sensor deployment.
• Database table name: The table/virtual sensor/file to which the data is being logged.
• Database parameter name: The parameter in the table to which the data is being logged (if more than one parameter is logged by the same sensor, the user must create multiple sensor deployment forms, one for each parameter).
• Parameter number: The order in which the parameter appears in the database.
• Measurement Media: The medium that is being measured (multiple values can be entered).
• Measured Parameter: The specific parameter being measured (e.g. temperature).
• Units
• Sampling frequency: The sampling frequency that has been implemented for this sensor deployment.
• Email address: the email address of the person responsible for the sensor
• Contact telephone number: the contact number of the person responsible for the sensor
• Organization: the organisation to which the sensor belongs
• Department: provided for better location within the organisation where appropriate

Once this has been filled out and 'save page' has been pressed, the user is taken to the sensor deployment page:

Query results on this page show links to the sensor deployed, the database parameters associated (or previously associated) with this sensor and the measurement locations with which this sensor is/has been associated.

Observations

Observations made in the field that cannot be directly related to invalid data with a defined start and end time should be entered here.

Observations links are provided on the measurement location page as well as on the database parameter page.

The form for entering observations includes the fields:

• Database table name
• Database parameter name
• Observer name: the email address of the observer
• Timestamp: date of the observation
• Observation: Text
• Image (optional)

It is important to note that if the measurement location is entered instead of the database table name and database parameter name in this form, the observation will be shown in a query on the measurement location page.

Pictures should be included as much as possible. To upload a new image, click Upload File to the right of the Image field (see the help box on this form for details about formatting and how to upload multiple images).

Filling out the form and clicking on 'save page' will create the following page:

Invalid Data

Invalid data is noted when analysing the data and seeing that the data is completely unreasonable. It can be defined by a start and end date. Invalid data forms can be created by clicking on the link on the database parameter page. The form for entering invalid data simply has the fields:

• Database table name
• Database parameter name
• Observer name: the email address of the observer
• From/to time: dates during which the data was invalid
• Description (optional)
• Image (optional)

Filling out the form and clicking on 'save page' will create the following page:

All invalid data for a given sensor is presented (as a query) on the sensor deployment page.

Editing a page

Any of the above information can be edited by navigating to the summary page, clicking on the 'edit' link, editing the desired field and clicking 'Save page'.

Deleting information

Any of the above information can be deleted by navigating to the relevant page, clicking on the 'edit' link at the bottom of the page, deleting all of the information and clicking 'Save page'. All links to the information in other pages will also be removed. Only admin has the rights to delete the page entirely.

For further help, please contact the project manager.