Swiss Experiment Backup

Swiss Experiment (2007-2011)

STILL UNDER CONSTRUCTION

My involvement: programme coordination, construction of wiki metadata system, construction of SLF data system with GSN, data management, construction of long-distance wifi links, initiation and field maintenance of Sensorscope deployments and other field deployments in Davos.

Swiss Experiment was an initiative of the ETH Competence Centre for Environment and Sustainability (CCES) to provide cross funding for environmental infrastructure projects and enable the application of new measurement techniques and technologies to environmental research, combining technological and environmental research in one place. Swiss Experiment particularly aimed to increase the availability of data at high temporal and spatial resolutions. This was a collaboration of 12 departments from across the ETH domain and UniZH. The funding from Swiss Experiment mainly contributed to existing projects with the aim of performing additional cross-disciplinary activities.

Data Infrastructure
Sensor Networks
Measurement technologies
Intensive Deployments
- Precipitation measurement campaign in alpine terrain
- Vispa catchment measurement campaign
Scaling of precipitation data
Education campaigns

SwissEx data infrastructure
My involvement: programme coordination, construction of wiki metadata system, construction of SLF data system with GSN, data management

The data infrastructure aimed to use the Global Sensor Networks (GSN) as a distributed network of live data repositories. These repositories were hosted at the research institutes such that data ownership questions could be satisfied. GSN simplified data acquisition through the use of data acquisition wrappers, which could be configured for a specific data type. By chaining multiple wrapper types, further data cleaning could be achieved, either allowing both raw and processed data to be saved, or just processed data. A model interface took this wrapper integration one step further by allowing models to be configured and run on real-time data from within the GUI. Processing was dispatched to dedicated model servers and the results returned to the web platform. Research also took place to try to automate data quality calculations for live data.
Both a web interface and an API were available for querying the data and notifications were configurable for events occurring in the data.
http://sourceforge.net/apps/trac/gsn

The data portal of WSL/SLF using GSN middleware

The GSN data portal of EPFL-EFLUM showing the La Fouly experiment

The control interface of for model integration in GSN. This shows a GeoTop simulation.

The semantic mediawiki served as a central platform for entering data and metadata and for querying this metadata together with the automated data collections from GSN including the results of any automated modeling.
The semantic mediawiki supported a web interface on top of a database, which could be queried using SPARQL. In addition to this, extensions were added to allow the use of the R programming language within the wiki. This allowed us to query both metadata and data simultaneously and plot the annotated data in the Wiki interface. Combined, this data source search functionality together with the data query interface, provided a centralised interface to the distributed data system.

The metadata discovery user interface, showing a timeline of events and a map of locations, with filters according to data types etc.

An example showing the availability of metadata in the system (left and middle) and the availability of data of a specific type (right)

An interface in the semantic mediawiki, mixing data queried from GSN with metadata from the mediawiki to highlight events in the data.

Both GSN and the Semantic Mediawiki were computer science research projects.

Bonvin N., Papaioannou T.G., Aberer K., Autonomic SLA-driven Provisioning for Cloud Applications, August 2011
Paparrizos I., Jeung H., Aberer K., Visualization and Tagging of Sensor Metadata, Proceedings of the IEEE International Conference on Data Engineering (ICDE), 3 April 2011
Paparrizos I., Jeung H., Aberer K., Advanced Search, Visualization and Tagging of Sensor Metadata, Proceedings of the IEEE International Conference on Data Engineering (ICDE), March 2011
Popescu A.D., Dash D., Kantere V. and Ailamaki A., Adaptive query execution for data management in the cloud, Proceedings of the second international workshop on Cloud data management (CloudDB), October 2010
Jeung H., Sarni S., Paparrizos I., Sathe S., Aberer K., Dawes N., Papaioannou T.G. , Lehning M., Effective Metadata Management in Federated Sensor Networks, 3rd IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (SUTC2010); California, USA, 7 June 2010
Jurca O.A., Michel S., Herrmann A., Aberer K., Continuous Query Evaluation over Distributed Sensor Networks, Proc. 26th IEEE International Conference on Data Engineering (ICDE 2010); Long Beach, California, USA, March 2010
Wombacher A., Schneider P., Observation Centric Data Model, Proc. 21st DEXA Conference, Bilbao, Spain, 2010
Jeung H., Yiu M.L., Zhou X. and Jensen C.S., Path Prediction and Predictive Range Querying in Spatial Network Databases, The VLDB Journal, August 2010
Haghani P., Michel S., Aberer K., The gist of everything new - personalized top-k processing over web 2.0 streams, Proceedings of the 19th ACM Conference on Information and Knowledge Management (CIKM), November 2010
Haghani P. , Michel S., Aberer K., Evaluating Top-k Queries over Incomplete Data Streams, Proc. 18th ACM conference on Information and Knowledge Management, November 2009
Zhou Y., Salehi A., Aberer K., Scalable Delivery of Stream Query Result, Proc. 35th International Conference on Very Large Data Bases (VLDB), July 2009
Salehi A., Riahi M., Michel S., Aberer K., Knowing When to Slide - Efficient Scheduling for Sliding Window Processing, Proc. 10th International Conference on Mobile Data Management (MDM), May 2009
Salehi A., Riahi M., Michel S., Aberer K., GSN, Middleware for Stream World, Proc. 10th International Conference on Mobile Data Management (MDM) (demonstration track), May 2009
Michel S., Salehi A., Luo L., Dawes N., Aberer K., Barrenetxea G., Bavay M., Kansal A., Kumar K.A., Nath S. , Parlange M.B., Tansley S. , Van Ingen C. , Zhao F. , Zhou Y., Environmental Monitoring 2.0, Proceedings of ICDE (2009), March 2009
Jurca O.A., Michel S., Herrmann A. , Aberer K., Processing Publish/Subscribe Queries over Distributed Data Streams, Proc. 3rd ACM International Conference on Distributed Event-Based Systems, 2009
Dawes N., Kumar K.A. , Michel S. , Aberer K. , Lehning M., Sensor Metadata Management and Its Application in Collaborative Environmental Research, escience, pp.143-150, 2008 Fourth IEEE International Conference on eScience, December 2008
Jurca O.A., Michel S., Herrmann A., Aberer K., Query Driven Operator Placement for Complex Event Detection over Data Streams, Proc. 3rd IEEE European Conference on Smart Sensing and Context, 2008
Sathe, S., Hoyoung Jeung, Aberer, K.; Creating probabilistic databases from imprecise time-series data; Data Engineering (ICDE), IEEE 27th International Conference pp.327 – 338, doi:10.1109/ICDE.2011.5767838 IEEE CONFERENCE PUBLICATIONS, 2011

For more advanced cross-network queries, the GSN nodes were accessed by two further research projects:

A collaboration with Microsoft Research to pull data into a 3D map interface - SensorMap
WIthin SwissEx, we collaborated with Microsoft research to push the boundaries of what was available for data discovery and visualisation. SensorMap allowed the display of both point data time-series in a plot as well as time-series raster data e.g. from simulations with a time control. The data was visualised on top of Bing Maps, which allowed multiple base maps and even display of the overlayed raster on the 3D topology (in 2008! Google maps was only launched in 2005). The experience of SensorMap was ahead of its time, even adding 3D visualisations of global cloud cover data. Unfortunately, SensorMap was discontinued by Microsoft Research before it had a chance to make an impact on the community.

The Microsoft Research SensorMap system showing some of the capabilities

Lehning, M., Dawes, N., Bavay, M., Parlange, M.B., Nath, S., Zhao, F.: Instrumenting the earth: next-generation sensor networks and environmental science; Chapter in The Fourth Paradigm: Data Intensive Scientific Discovery, 2009, doi: 10.1109/jproc.2011.2155130. http://research.microsoft.com/en-us/collaboration/fourthparadigm/

A WebGIS developed by the ETHZ Cartographical Institute - GeoVITe
At the same time as we were working on SensorMap, we also had a collaboration with ETHZ-IKG on the GeoVITe platform. The focus of GeoVITe was on sharing Swiss spatial data, which meant that it provided SwissEx users with an interface for accessing Swisstopo data as well as other national data sources (vector and raster). Unfortunately, due to content licencing restrictions at the time, this also meant that usage was restricted to the ETH domain. GeoVITe had much more of a cartographical focus than SensorMap, and conformed to web standards, meaning that it was able to access many more cartographical data sources.
Later (in the OSPER project), this interface was also extended to enable access to the GSN-based sensor data.

The ETHZ-IKG GeoVITe user interface

Iosifescu I. and Hurni L. (2010). GIS Platform for Interdisciplinary Environmental Research, in Proceedings of the 7th ICA Mountain Cartography Workshop, Borsa, Romania

SensorScope 2nd and 3rd Generation Networks
My involvement: feedback from deployments
SensorScope was a successful attempt to build low energy, plug and play, wireless self-configuring local networks of stations. This meant that the stations could communicate data locally at low power, and only needed one higher powered base station with a mobile connection to communicate the data back to the server. SensorScope became a successful spinoff of EPFL. The SwissEx collaboration provided significant funding both directly and in terms of orders to develop the SensorScope system.

SensorScope extreme environment test deployments
My involvement: management of field deployments in Davos
In order to field test the SensorScope stations in extreme environments whilst also contributing to scientific experiments, two experiments were set up, one in La Fouly, Valais, and one on Wannengrat, Davos. These experiments both ran for many years and provided vital feedback to the development of SensorScope, resulting in more robust and power efficient hardware.

SensorScope device pool
My involvement: acquisition, management and deployment of the SwissEx equipment pool
A pool of SensorScope devices was purchased and used for short-term (e.g. 1 year) deployments in various CCES projects

Long distance communication links
My involvement: research, implementation and testing of long distance links (incl. custom router builds)
Data communications in 2008 were different to today. Mobile data communication was slow and expensive and coverage, though good, was still limited in comparison to today. In order to overcome these hurdles, I built WiFi communications links with directional WiFi antennae. In the case shown below, the router was given GSM capabilities and provided mobile hotspot access to a valley where mobile coverage was unavailable.

Creation of a wifi hotspot across the valley floor using a directional antenna

The mobile links were used for many years in Davos in a multi-hop scenario, providing high-bandwidth data communications to several field-sites, one of which was not in a direct line-of-sight. These links were used to acquire acoustic (geophone) data for the TRAMM project.

Permasense 2nd Generation Network
My involvement: management of SwissEx and hence collaboration on shared data acquisition technologies
In the Permasense 2nd generation network, the SwissEx collaboration provided Permasense with the GSN data acquisition system as well as long distance wifi links to communicate the data from the Matterhorn, across to the Kleine Matterhorn. As of 2025, these are still in operation.
http://data.permasense.ch/data.html#data

In turn, Permasense technologies were also deployed to SwissEx partner experiments, RECORD for groundwater monitoring.
Images taken during a SwissEx excursion to the permasense site on the Matterhorn:

Hydromon self organising wireless network
My involvement: none
Hydromon aimed to create a sensor network for monitoring the drinking water distribution system of a major agglomeration in southern Switzerland. A sensor network was deployed where several tens of parameters (physical and chemical) and five pathogens are constantly monitored 24/7 at each node of the network. SwissEx provided academic funding in a collaboration between EPFL and a commercial company, which became an operational system.

X-Band Radar Precipitation Measurements
My involvement: assistance in deployment and maintenance
SwissEx was the start of a long-term collaboration between the group of EPFL LTE and WSL/SLF for joint precipitation experiments using X-Band radars and intensive ground-based measurement campaigns. During SwissEx, the newly acquired dual-polarized X-Band radar was installed on Jakobshorn in Davos, across the valley from the Wannengrat field site and Sensorscope measurement campaign. Since this time, similar experiments have taken place both in Davos and elsewhere.
The dual-polarized X-Band radar allowed new finer resolution measurement of precipitation and indication of particulate dimensions in two axes.

The dual-polarized X-Band radar installed at Brämabüehl, Jakobshorn, Davos, on the opposite side of the valley to the Wannengrat field site, for precipitation experiments

Optical disdrometer development
My involvement: none
EPFL-ISIM together with EPFL-LTE pushed the boundaries of disdrometer development by manufacturing a 1k pixel single photon camera which could operate at 6kfps continuously. This meant that individual water droplets could be analyzed for size and shape, creating an optical disdrometer.

An initial prototype of the disdrometer hardware

A snapshot of the disdrometer output, showing a single droplet measured in real-time

The disdrometer main board showing the single-photon optical detector in the middle

Thermistor-based anemometer testing
My involvement: none
Thermistor based sensing technologies developed for other projects were compared to sonic anemometer measurements to analyze the feasibility of cheap anemometers without moving parts.

A thermistor-based detector built for a different experiment was tested for its potential as an anemometer.

Thermistor-based wind-speed calculation was compared to a 3D anemometer

Monitoring of snow height and snow water equivalent with GPS
My involvement: assistance in field deployment and provision of data
10.13140/RG.2.2.11896.29442

Development of Distributed Temperature Sensor (DTS) calibration techniques for field deployment
My involvement: assistance in logistics
DTS allows continuous temperature measurements at discrete locations along an optical fibre. To do this, the temperature at the incoming and outgoing ends of the fibre must be known in order to infer the temperature at locations between these calibration points. Researchers at EAWAG developed calibration techniques for DTS equipment which could be deployed in the field.

LIDAR field campaign and development
My involvement: none

Precipitation measurement campaign in alpine terrain
My involvement: none

Vispa catchment measurement campaign
My involvement: none

Scaling of precipitation data
My involvement: none
ETHZ-HRWM carried out temporal scaling studies on precipitation data and determined that whilst precipitation cannot be perfectly scaled, there are distinct regimes where scaling can be successful, connected to atmospheric processes.
Paschalis, A., P. Molnar, and P. Burlando, (2012) Temporal dependence structure in weights in a multiplicative cascade model for precipitation, Water Resour. Res., 48, 1, doi:10.1029/2011WR010679.

Education campaigns
My involvement: none
Educational campaigns were regularly carried out in the areas of climate and seismology, with cross-funding provided by SwissEx. These campaigns took place in schools across Switzerland as well as in Burkina Faso.