From SwissExperiment
Infrastructure
The SwissEx Infrastructure
What IS the 'SwissEx' infrastructure?
The infrastructure is designed to allow scientists to publish data in a world leading intelligent infrastructure, where data is made available (with or without access control) over the web, metadata is not only stored, but both manually AND automatically produced and made available with the relevant data and distributed databases can be combined into federated sensor networks - part of the rapidly evolving Sensor Web.
The infrastructure built up under SwissEx is not a 'SwissEx infrastructure'
i.e. it is not limited to projects running under SwissEx, but is an open source, cutting-edge infrastructure, designed to be reproduced and used throughout the world by various collaborations of institutions.
What benefits does the SwissEx infrastructure bring to the scientist?
The infrastructure aims to store data in a recoverable fashion. How many PhD students produce fantastic data sets, which get lost on a filesystem where noone knows it exists let alone the situation in which it was recorded? The latest computing infrastructures can process and store data in a way that it is searchable based on metadata or data, useable by third parties and accessible using a variety of methods which can be integrated into third party tools.
What are the tools that provide this?
A flow chart is provided below to show the steps and choices in the SwissEx infrastructure
Infrastructure Usage
More on how the infrastructure is designed to be used (also found in the Data Management menu):
SwissEx Infrastructure Example Use
A real example of how this infrastructure can be used is found in the BigLink project. The runoff measurement instrument consists of a collection receptacle which must be emptied by the field engineer once it exceeds a certain level.
Previous to SwissEx, the liquid volume data was automatically returned to a ‘Loggernet’ instance which laid down the data into a csv file. The field engineers did not have access to this server and had to regularly call the project manager in order to check on the level of liquid in the bottle.
With SwissEx in place, the csv file will be read by GSN and stored in the GSN database. This means that the csv file may be overwritten on each update. Through either GSN or the SwissEx wiki, the field engineers may view the data online. An algorithm will be included in the virtual sensor which will check whether the data has been consistently above a threshold (to avoid false alarms) or possibly will check that the runoff has increased consistently until it exceeds the threshold. GSN will then send a SMS/text message to the field engineer to tell him/her that the runoff measurement equipment requires attention. The field engineers can then check on the calendar in the wiki to find out if anyone is going to the site in the near future and ask them to empty the receptacle. On emptying the receptacle, an observation is entered into the Metadata wiki entry pages that this work was carried out on this day at this time.
Through GSN and SensorMap, the analyst will be able to plot the runoff data against the meteorological station data (surface temperature, precipitation, air temperature) and e.g. may perform correlations to find the lag between the meteorological parameters and the increase in runoff volume. This data will be tagged according to the quality of the data as well as having tags to inform them that the receptacle had been emptied at a certain point, hence the inconsistency in the data. This eliminates the requirement for the analyst to go back to the field engineer’s logbooks and try to work it out for himself. When modelling, the model can be set to ignore any values which have a bad reliability tag, e.g. large jumps in values or very constant values which may indicate that the instrument is stuck. This saves the manual editing of data prior to entry into the model.
