Automatic validation of sensor data

anomaly detection by Sensative Machine Learning

Sensors from SMHI (Sveriges Meteorologiska och Hydrologiska Institut) measuring water flow in rivers in Norrland have problems with ice formation. Therefore, they often report incorrect values ​​during wintertime. Raw sensor data is stored in SMHI databases, and a program that identifies anomalous data is run. A meteorologist then reviews and approves the data, after which it is called qualified (or validated) and stored in a database of ‘qualified data.’ Only qualified data may be used for statistics, weather models, and environmental calculations to avoid erroneous results. 

Agriculture is about to become data-driven, and it is then necessary to qualify raw sensor values ​​before it can be used as a decision basis in various models for plant protection, nutrient supply, harvest calculations, etc. There will be no possibility for people to overlook the massive amount of data that is generated monthly or annually in the way done by the meteorologists at SMHI. 

Modern ML-system (within ‘deep learning’) can now qualify content better than humans, e.g., for review of images, ML systems create 2.5% errors on average, compared to 5% created by people. Such methods are used in this project and manage multiple neighboring sensor systems to find errors and correct and replace incorrect data from individual sensors with estimated reasonable values, in a similar way that people act to validate data for calculation models to be run. 

The project will identify common sensor problems for the most interesting sensors in agriculture, including sensors for temperature, humidity, precipitation and wind in weather stations, sensors for soil temperature and humidity, etc. 

The project will develop models and algorithms (machine -learning, ML) and train these with previously collected data from agriculture (e g via LantMät administered by Jordbruksverket). 

The trained models will then be tested against data collected during the course of the project, in order to deliver added value for web-based decision support, cultivation systems , etc.

Sensative part: Sensative contributes to the project primarily with expertise regarding data analytics and machine-learning (ML), but also with the Yggio data interoperability platform, sensors and generic IoT expertise, for development of models, algorithms, functional libraries and services for validation of sensor data, analysis of various data sources and combination of these sources to provide extensive values for agricultural applications.

Participants: Human Fidelity, Jordbruksverket, Luda.Farm, Sensative, SLU, Videocent

External funding: the European Innovation Partnership for Agricultural Productivity and Sustainability (EIP-AGRI) with support from Jordbruksverket

Europeiska jordbruksfonden jordbruksverket

Duration: April 2019 – December 2021

Visitor survey

Visitor survey

How can we get better?

Please tell us a bit about yourself and why you visit so we can provide you with better information in the future.

Thank you for subscribing to the Sensative newsletter

Your PDF download is now unlocked

Also, we have unlocked all services exclusive for the Sensative community