J.C. Whittier successfully defends dissertation proposal

Today, J.C. Whittier successfully defended his dissertation proposal “Towards an efficient, scalable stream operator framework for computing continuous fields”, and advanced to Ph.D. candidacy.

Congratulations, J.C.!


SIE556 Real-time Sensor Data Streams Final Projects (2016)

This semester, the students again came up with a great selection of final projects, applying their new knowledge of real-time sensor data streams processing with mysql, python stream queries and Cloudera’s Spark and some Arduino programming to a project of their interest.

There are several general thematic groups in the class:

Group 1: Using Arduinos and temperature and humidity data to monitor environmental conditions in real-time

Jon Cole built a Unity-based 3D Visualization of live and historic buoy data in the Gulf of Maine.

Check out the interactive demo here. Code here.

Devon Stetson built a sensor data stream analysis system deploying a temperature data in his dorm room.

Welles Tisdale extended his undergraduate capstone project of building an automated greenhouse using Arduinos with live sensor data analysis, alerts and long-term monitoring.

Group 2: Real-time smartphone sensor data analysis to detect parking and departing events at UMaine in real-time.

Avery Dunn and Anthony Stetson’s current capstone project of real-time smartphone sensor data analysis to detect parking and departing events at UMaine in real-time

Group 3: Real-time Speeding Alerts using Arduinos and a GPS unit.

Kaitelyn Haase has just started to learn programming at the beginning of this semester as a graduate student in the MSIS program, and so it is even more impressive that she managed to soldered an Arduino “sandwich” with a shield and an GPS unit, program the Arduino to compute the current speed in real-time and sound a audio alert when the speed was over the speed limit of 70Mi/H. She also analyzed and visualized the collected information.

Group 4: Interactive Visualization of Argo Drifter Data.

Brad Sheperd built a python-based graphic user interface to stream process netCDF encoded stream data from the Argo drifters that cover the oceans world-wide. The visualization allows to interactively spatially subset the data and select different parameters.

Group 5: Transforming Sensor Data into Sounds using SuperCollider

Rod O’Connor combined a set of light sensors on an Arduino with python stream based processing the incoming data and sending it to SuperCollider to ‘animate’ the sensor readings as sound (very cool!).

Interested in learning Arduino programming?

For a new section of COS120 Introduction to Programming, we made a few accompanying videos for the Sparkfun Arduino starter kit. Katie Manzo, a CS undergraduate, is my learning assistant for the course, and she is demonstrating the kit.

Here’s to setting up:

This is the first example, the blinking LED:

This is another example, the photo resistor:

This is the 4th example, spinning a motor:

Congratulations, Joel!

Today, Joel Whitney presented his graduate project work on the implementation of a temporal field class library, and his internship at ESRI in Portland this summer, and with this fulfilled the remaining requirements for his MS program.

Joel joined us in the spring 2015 semester for the MS in Spatial Information Science and Engineering program. For the blueberry soilmoisture sensor network, he also has been our liaison with Cherryfield Farms, where he worked before coming back to Umaine for his MS degree.

The occasion was celebrated with some beers and pizza at Mason’s Brewery in Brewer.

All the best, Joel!



Harlan, Joel, Tega, JC, Kate (from left to right)

Implementing a Temporal Field Java library and QGIS plug-in

Over the last 6 months, Joel Whitney developed a Temporal Field Java library that wraps sensor data streams, which are either stored in RDBS, CSV files or are live streams, as Temporal Fields. A temporal field wraps an individual sensor data stream, and makes the sampling frequency transparent via its interface. It allows a user to request a value at any point in time (over which the temporal field is defined). The Temporal Field retrieves the available sampled data around the query point and can either return the value directly or estimates the value based on actual sampled data on the fly. The user can specify the temporal range around the request data point that should be used for estimation.

Wrapping streams as temporal fields make it easier to integrate many individual sensor data streams and ‘cut across’ them at user-defined snapshots.

In addition, Joel implemented a QGIS extension, the Temporal Field gateway plug-in. It creates a call to a Temporal Field Object, and allows the user to parameterize the data of interest, and add it to a QGIS application.

For more details, please watch the following videos:



Hadoop Summer Camp

Continuing with our ‘let’s learn something new’ summer camp from last year, this year we focus on learning Hadoop. We installed Virtual Box, and the Cloudera quickstart engine, and used the first tutorial as our “hello World” program option, with various forms of success. 8GB of RAM and 7GB of RAM associated with the VB is definitely a recommended start to not twiddle fingers for Cloudera to respond.


J.C. our instructor


This year we had to break out into the class room


Our camp crew: professors, grad students and 1 CS undergrad



Our paper “From Data Streams to Fields: Extending Stream Data Models with Field Data Types” accepted at GIScience 2016

Our paper “From Data Streams to Fields: Extending Stream Data Models with Field Data Types” by Qinghan Liang, Silvia Nittel and Torsten Hahmann was accepted in the full paper track at GIScience 2016.


With ubiquitous live sensors and sensor networks, increasingly large numbers of individual sensors are deployed in physical space. Sensor data streams are a fundamentally novel mechanism to create and deliver observations to infor- mation systems, enabling us to represent spatio-temporal continuous phenomena such as radiation accidents, pollen distributions, or toxic plumes almost as instan- taneously as they happen in the real world. While data stream engines (DSE) are available to process high-throughput updates, DSE support for phenomena that are continuous in both space and time is not available. This places the burden of handling any tasks related to the integration of potentially very large sets of con- current sensor streams into higher-level abstractions on the user. In this paper, we propose a formal extension to stream data model languages based on the concept of fields to support high-level abstractions of continuous ST phenomena that are known to the DSE, and therefore, can be supported through queries and process- ing optimization. The proposed field data types are formalized in a data model language independent way using second order signatures. We formalize both the set of supported field types are as well as the embedding into stream data model languages.


Full paper can be found here:Liang_Nittel_Hahmann_GIScience2016

%d bloggers like this: