Data management and workflow

We love helping experts in niche fields to add value to very large volumes of marine data. We create systems that reduce administrative overhead, reduce the need for training in “the computer system”, and help to locate and correct data problems or prevent them occurring in the first place. With the time-consuming minutiae dealt with automatically, experts are free to focus on solving higher level business or research problems. We can also increase the value of data by giving confidence in its provenance.

How we can help

We’ve developed custom systems to help with

  • Bringing together a wide variety of information, such as sensor output, PDF documents, images, chart data, and so on
  • Tracing outputs back to the source of the raw data, including all the corrections, updates, reviews, checks, and calculations that contributed to the final result
  • Visualising data to help expert staff with analysis
  • Managing the flow of tasks around different members of staff in an organisation
  • Producing print-ready books and reports, as well as automatic online data distribution
  • Automatically identifying potential data problems
  • Simplifying the correction of data problems, for example, sensor clock drift

Publishing

We know that publishing data can be a fiddly job, and that it doesn't always come out quite right first time. We know how hard it is to manage a team of people trying to add new data while someone else is trying to make last minute corrections to a publication. We can give you full control over exactly what subset of your data will end up in a publication and who can change that subset at any given time.

Data Provenance

Many organisations that gather data struggle with understanding the provenance of all the data in their system. What was the accuracy of the instrument that gathered it? Are there problems in the data caused by sensor errors? Or have the problems been removed, and how does one verify that they were removed correctly? How can one effectively prevent "dirty" data from being accidentally used in calculations?

We can create systems that allow users to navigate from the outputs of calculations all the way back to details of the sensor that gathered the data, and the exact raw data that was gathered. Users can easily see the corrections and adjustments that have been applied to the raw data. We can introduce workflow that requires all changes to be checked by another person.

We can also provide tools that can be used at a remote site to catalog gathered data immediately, so that it can be correctly added to the system days, weeks or even months later.

An example

Sarah collects data from Tide Gauge 3638. Later that week when she is moored somewhere with wifi, she uploads her gathered data into the system. Ben is in the main office, and sees that the system has flagged up a day where the tidal heights suddenly jump by 3cm. Ben gives Sarah a call; she arranges a detour back to the tide gauge and finds that indeed it has been knocked and is 3cm from its intended location. Sarah repositions the gauge. Meanwhile Ben applies a 3cm adjustment to the erroneous section of the data and marks the data as pending Amal’s approval.

The dataset appears on Amal’s work list. She reads Ben’s note about the correction, and brings up the corrected data on a graph over the raw data and the predicted tide heights for the same period, to satisfy herself that the correction is reasonable. She marks the corrections as “approved” and attaches Sarah’s report on the tide gauge fault. The corrected data will now be taken into account next time tidal harmonics are generated.