From SwissExperiment

Metadata Management User Guide

Your data is streaming through GSN into SensorMap and you have users at several sites observing this data and using it for different purposes, if your data is public, you may not know who your users are.......You go into the field, repair a broken sensor, replace another and notice that another sensor is not working as it is frozen.....how do you communicate this to the users? If you merely write it in a notebook, they will never know about it.

The SwissEx metadata management wiki extensions are designed to aid projects in creating a metadata database on the hardware used in their experimental fieldwork as well as on the site itself. Data is entered using a user interface designed to guide the user in entering hardware information and observations. The semantic wiki then automatically stores this metadata in a SQL database so that it may be queried along with the corresponding data.

The database created using the SwissEx extensions may be read by either the SPARQL GSN wrapper or directly by SensorMap to provide information entered through the wiki interface directly where it is needed: alongside the data.

Contents 1 Metadata Management User Guide 1.1 Fieldsites 1.1.1 Entering a fieldsite just for the purpose of the fieldsite database 1.1.2 Entering a fieldsite with the purpose of entering further metadata on the site 1.2 Measurement Locations 1.3 Sensor models 1.4 Sensor Instances 1.5 Transfer Function 1.6 Experimental Method 1.7 Sensor Deployment 1.8 Observations 1.9 Invalid Data 1.10 Editing a Station, Model, Sensor or Action 1.11 Deleting a Station, Model, Sensor, Parameter, Invalid data period or Observation

Fieldsites

The main entry point for the metadata is through the fieldsites database: http://www.swiss-experiment.ch/index.php/Fieldsite:Home (accessible through the 'data' link on the front page). This provides a map/list of all fieldsites entered using the forms provided.

A fieldsite can be added using the form provided here info.pngchange the URL if you wish to have a more comprehensible link .

Entering a fieldsite just for the purpose of the fieldsite database

This process has been made easier by automatically generating a page so that you just have to enter the information and click 'save page'

Click on the link to automatically generate a page containing the form for entering data. This will require you to be logged in. To register please contact the admin.

Enter the information as prompted by the form.

• Deployment Name: the name of your fieldsite
• Image link: follow the link provided to upload your photograph.
• Description: this should be searchable keywords describing what is measured at the site.
• Coordinates: enter the coordinate into the box below the map, press 'look up' and the coordinates will be inserted into the box above the map in the correct format. Alternatively, click on the map to select your coordinates.
• Altitude: the altitude of the fieldsite in meters
• Swiss Latitude/Longitude: the swiss latitude and longitude (Northings and Eastings) of the fieldsite.
• Start/End date: the dates from and until which the fieldsite was used to collect data
• Institute responsible: the name of your institute so that people may get in contact if they are interested in the site. This is purposefully left without a specific contact. If you are in a large organisation, you may wish to specify your department
• Free text: please use this to describe the fieldsite in more detail, including exactly what is/has been measured there

On clicking save, the user will be presented with a table containing all of the information added plus a map of the location.

Once a fieldsite has been added, the URL should be sent to the wiki administrator so that it can be made public.

Entering a fieldsite with the purpose of entering further metadata on the site

Once the link has been clicked, the user is presented with a box, into which they should insert a namespace and pagename*. Following the specification of this pagename, the user will be presented with a form, into which the relevant information should be added before clicking 'save page'.

The following fields are provided in the form:

• Deployment: the name of your fieldsite
• Image link: follow the link provided to upload your photograph (this link can be found at the bottom of every page when logged in), then insert the last part of the URL (Image:filename.jpg) into the box, enclosed by double square brackets

  [[Image:filename.jpg]]

• Description: this should be searchable keywords describing what is measured at the site.
• Coordinate: enter the coordinate into the box below the map, press 'look up' and the coordinates will be inserted into the box above the map in the correct format. Alternatively, click on the map to select your coordinates.
• Swiss Latitude/Longitude: the swiss latitude and longitude of the fieldsite.
• Swiss Altitude: the swiss altitude of the fieldsite.
• Altitude: the altitude of the fieldsite in meters
• Start/End date: the dates from and until which the fieldsite was used to collect data
• Institute responsible: the name of your institute so that people may get in contact if they are interested in the site. This is purposefully left without a specific contact. If you are in a large organisation, you may wish to specify your department
• Additional Information: please use this to describe the fieldsite in more detail, including exactly what is/has been measured there

*A namespace is a delimiting system within the wiki. We are no longer providing namespaces for projects or field sites. If you already have your own namespace in the wiki, you may use this, otherwise use the Fieldsite namespace (enter 'Fieldsite:MyFieldsiteName' into the box).

On clicking save, the user will be presented with a table containing all of the information added plus a map of the location and another map/table which will contain the measurement positions once they have been entered.

Once the fieldsite page has been produced, it will automatically be shown in the table and map on the fieldsite database page (these are simply formed by queries to the database). The fieldsite page can be extended to provide any additional information that you like (not in database form). To do this, simply click on the edit link at the bottom of the page. The information created by the forms is shown, enclosed in curved brackets{}. This information should not be disturbed, but you can add additional information using wiki syntax above or below the block of information enclosed by the brackets.

The fieldsite page shows the results of a query. This query returns the measurement locations which are registered to this fieldsite. The results are displayed in map and table form.

Measurement Locations

The measurement positions are positions, such as fixed stations or revisited sites at which measurements are taken within the fieldsite.

If the second type of form was used (that providing the full range of metadata functionality), there will now be a link provided to add measurement locations. This again requires the user to enter the page name as the Namespace followed by the name of the page (usually the measurement location name).

Enter your measurement location information and click on 'save page'. You will be presented with the measurement location page and a link will be automatically placed in the measurement locations table and map in the fieldsite page.

The measurement locations page shows the results of a query that shows the sensors and database parameters corresponding to each sensor deployment at that location (incl. dates) and the database table name.

THE MINIMAL USER CAN STOP HERE, JUST HAVING A FIELDSITE PAGE AND MEASUREMENT LOCATION PAGES. OBSERVATIONS CAN BE ADDED TO THE MEASUREMENT LOCATION PAGE USING THE OBSERVATIONS SECTION BELOW. IF THE FIELDSITE CONTAINS MANY LONG TERM SENSORS, THE USER SHOULD READ ON.

Sensor models

Sensor types can be identified according to the model number and manufacturer. This information is stored along with specific information about the model in the sensor model page.

A sensor model can be registered by clicking on the link on the measurement location page

Specific information desirable for describing the sensor model include:

• Lower range: the lower bound of measurable values from the sensor
• Upper range: the upper bound of measurable values from the sensor
• NaN value: this can be used by GSN to convert these values to empty cells in the database, hence providing gaps in the data instead of unknown values
• Measurement error: the accuracy to which the value can be measured
• Decimal places: the number of decimal places provided by the instrument

Once 'save page' is pressed, the following page is shown. The table shows the results of a query for the instances of this sensor model registered at the fieldsite.

Sensor Instances

A sensor instance is a specific sensor which can be identified by its serial number.

A sensor instance can be registered into the system using the link on the measurement location page.

On clicking the link, the user is presented with a form containing two parameters (Sensor Model and serial number):

Once completed and saved, the system returns a page with a number of query results:

• Sensor model: the link to the sensor model and its parameters
• Transfer function: transfer functions relating to this sensor instance
• Database parameters: database parameters fed from this sensor
• Sensor deployments: deployments of this sensor instance within the fieldsite (there may be more than one entry, each corresponding to a database parameter. A database parameter may be related to multiple sensor instances over time)

Transfer Function

A transfer function relates to a function combining all types of calibration which can be applied to the sensor in order to interpret the measured values into a true parameter value.

A transfer function can be registered using the link on the sensor instance page.

Multiple transfer functions can be recorded for each sensor instance, along with a corresponding date. It is assumed that a transfer function is valid for a sensor from the date recorded on the form until the next available date corresponding to the new transfer function.

The form has simply 3 fields: serial number, transfer function and date

Once this for is saved, the transfer function page is shown, which simply records the function

Experimental Method

The experimental method allows you to record exactly how the measurement was taken, e.g. if you have used a handheld instrument: how did you use the instrument, how did you insert the sensor etc. etc. i.e. everything that is required to understand a non-standardised measurement.

The experimental method is entered by the following form (accessed via the database parameter page)

The fields required on the form are as follows:

• Database table name: as for the database parameter table, i.e. the virtual sensor name for GSN or a file link for an excel file
• Database parameter name: the parameter within the virtual sensor/spreadsheet
• Experimental method name: a name to distinguish this method in the list of experimental methods
• URL reference: in case the experimental method exists somewhere on the web
• Literature reference: a reference for if the experimental method is described in published literature
• Method property: this is a link to a page within the wiki where you can describe your experimental method, e.g. Wannengrat:Home (the last part of the wiki page URL)

Once 'save page' is pressed, the page will look as follows, with a query showing all available experimental methods.

Sensor Deployment

The sensor deployment pages are the 'glue' to hold the system together.

Once you have created your sensor models and their various sensor instances and have set up your database parameters, you are ready to go into the field and deploy your sensors. When you deploy a sensor to a measurement location, note down the sensor's serial number and put it into this page. When you go to replace that sensor at a later date, you can fill in the end date of your sensor deployment and fill out another form to deploy a sensor at that point. Both of these sensors will be deployed to the same location and will feed the same database parameter, so when you open the database parameter, you will find which sensors were used between which times, hence which transfer functions should be applied to the data.

Using a different approach, if you are replacing both sensors and loggers/nodes, you will find not only which sensors were measuring at a specific measurement location between which times, but also which logger this has been logged through, hence you can join the data streams from two existing logger virtual sensors to create a virtual sensor corresponding to a measurement location.

By clicking on the deployment link on the measurement location page, the user is presented with the following form:

This form contains a number of fields:

• Measurement location: the measurement location to which the sensor/logger is deployed
• Sensor Instance/Serial No.: Unique identifier of the sensor
• Sampling frequency: the sampling frequency that has been implemented for this sensor deployment
• From/to date: the dates between which the sensor/logger is deployed at this position
• Relative X,Y,Z coordinate: the relative coordinates in comparison to the logger. If on a flat plane, these are +ve X=East, +ve Y= North, +ve Z=up. This coordinate system should always be referenced to the logger and Z should always be height/depth.
• Database table name: the table/virtual sensor/file to which the data is being logged
• Database parameter name: the parameter in the table to which the data is being logged (if more than one parameter are logged by the same sensor, the user must create multiple sensor deployment forms, one for each parameter)
• Parameter number: this value is required by SensorMap. It is the order in which the parameter appears in the GSN virtual sensor, where the parameter appearing at the top is parameter 0 and number sequentially as you read down the page.
• Measured Parameter: the real measured parameter (the parameter name may be an abbreviation)
• Measurement Media: e.g. the measured parameter of a thermocouple is temperature, the measurement media could be, for example soil or air.
• Unit
• Person Responsible: the email address of the person responsible for the sensor
• Contact Number: the contact number of the person responsible for the sensor
• Organization: the organisation to which the sensor belongs
• Department: provided for better location within the organisation where appropriate

Once this has been filled out and 'save page' has been pressed, the user is taken to the sensor deployment page

This page contains the results of a number of queries:

• Observations: observations of issues relating directly to the parameter
• Invalid data: observations of periods during which the data is invalid
• Experimental method: the method by which the parameter is measured
• Links to the sensors deployed.

Observations

Observations are things found in the field which cannot directly be related to invalid data with a defined start and end time.

Observations links are provided on the measurement location page as well as the database parameter page.

The form for entering observations simply has the fields

• Database table name
• Database parameter name
• Observer name: the email address of the observer
• Timestamp: date of the observation
• Observation: Text

It is important to note that in this form, if the measurement location is entered in place of the database table name and database parameter name, the observation will be shown in a query on the measurement location page

Both observations and invalid data applying to a parameter and to the measurement location are queried and displayed by Sensormap when a parameter is plotted.

Pictures should be included as much as possible. These are entered by uploading the image, copying the last part of the URL Image:filename.jpg and inserting it into the free text box, enclosed in double brackets [[Image:filename.jpg]] (to make the image smaller, left justified etc. you can use the following: [[Image:filename.jpg|400px|left]]).

Filling out the form and clicking on 'save page' will create the following page.

Invalid Data

Invalid data is noted when analysing the data and seeing that the data is completely unreasonable. It can be defined by a start and end date. Invalid data forms can be created by clicking on the link on the database parameter page. The form for entering invalid data simply has the fields:

• Database table name
• Database parameter name
• Observer name: the email address of the observer
• From/to time: dates during which the data was invalid

Filling out the form and clicking on 'save page' will create the following page.

All invalid data forms will be queried and displayed by SensorMap when a parameter is plotted.

Editing a Station, Model, Sensor or Action

Any of the above information can be edited by navigating to the summary page, clicking on the 'edit' link, editing the desired field and clicking 'Save page'.

Deleting a Station, Model, Sensor, Parameter, Invalid data period or Observation

Any of the above information can be deleted by navigating to the summary page, clicking on the 'edit' link, deleting all of the information in the fields and clicking 'Save page'. All links to the Station/Model/Sensor/Action in other pages will also be removed.

For further help, please contact the administrator or project manager