In this project a simple temperature and humidity monitoring system of a python snake enclosure was developed. The temperature and humidity values were read using a smart gadget from Sensirion (description). This device was connected via bluetooth to a Raspberry Pi 3 (RPi3) that stored the data in a MongoDB (version 2.4.14). A dash powered layout running on a flask server on the RPi3 is then used to display the stored temperature and humidity.

To document the complete code here would be impractical and difficult to follow. Yet, the working principle is outlined and a few key functions are elaborated on. The full code can be found in this repository: https://github.com/kromerh/paulisasnake

Data acquisition

Temperature and humidity data from the smart gadget is read with a frequency of 0.1 Hz (every 10 seconds). However, not all timestamps contain both temperature and humidity readings, which has its reason somewhere in the workings of the smart gadget that was not further checked into. The smart gadget stores the data in its memory and every 5 minutes the memory is read out and cleaned. Data is stored in the Raspberry Pi 3 in a mongo collection.

Script that runs with a cron job on the RPi3: https://github.com/kromerh/paulisasnake/blob/master/Sensirion_smart_meter/read_sensirion_and_insert_to_mongo_V2.py

Data cleaning

A few data cleaning steps have to be taken, i.e. removal of NaN’s which is straightforward with pandas. A bigger issue is the amount of data. For many periods of time, the temperature or humidity does not change at all, hence the following function is executed with a cron job every 10 minutes on the RPi3:

def filter_data(delta, data, col):
    """
    Filters the dataset by keeping only those datapoints that deviate more than delta from the
    previous one.
    Returns a pandas dataframe
    """
    data_return = pd.DataFrame()
    data_return['time'] = data['time'].copy()
    data_return['utc_time'] = data['utc_time'].copy()

    # copy temp or humid
    data_return[col] = data[col].copy()

    # shift by 1 and -1
    data_return[f'{col}_p1'] = data_return[col].shift(-1)


    # create mask if the value is larger than the delta compared to the value before and after
    data_return['diff'] = np.abs(data_return[f'{col}_p1'] - data_return[col])

    data_return['mask'] =  (data_return['diff'] >= delta)
    data_return = data_return[data_return['mask']]

    data_return = data_return[['utc_time','time',col]]

    return data_return

This function ensure that values are kept in the collection only if they differ more than some threshold (0.05 degC, 0.1 rel humid) from a previous reading. That way the display of the data does not take too long on the RPi3.

The code for this processing step is found in: https://github.com/kromerh/paulisasnake/blob/master/process_readings.py

Data visualization

Data visualization is done using a flask server that runs dash. The layout is found in this file: https://github.com/kromerh/paulisasnake/blob/master/dash/layout_base.py

The callback functions are in this file: https://github.com/kromerh/paulisasnake/blob/master/dash/callbacks.py

The image below shows how the layout of the monitor looks like (it is only accessible in the local network):

At the very bottom of this document, there are some instructions on how to install bluetooth and MongoDB on a RPi3.

Bluetooth on a RPi3

Link: https://tutorials-raspberrypi.de/raspberry-pi-3-wlan-einrichten-bluetooth/

Installation of MongoDB on RPi3 with Python3

• Install the packages with sudo apt-get install mongodb
• Create directory /data/db, be sure to check permissions correctly with sudo chrown -R mongodb /data/db/
• Start mongodb service using sudo systemctl start mongodb.service
• Check that it is also running with a restart of the system
• Install python packages python3 -m pip install pymongo==3.4.0 pandas numpy

MongoDB access and crontab

• the data from the test sensor is stored in the database sensirion_test, collection is temp_and_humid.
• to check, connect to the RPi3 and run mongo. Then run use sensirion_test and find everything in the collection using db.temp_and_humid.find().sort({$natural:-1}).
• execute read_sensirion_and_insert_to_mongo.py every minute in crontab adding * * * * * python3 /home/pi/paulisasnake/Sensirion_smart_meter/read_sensirion_and_insert_to_mongo.py to crontab -e