BME680 using the official Bosch Sensortec BSEC Library

ferazambuja · May 23, 2018, 4:03am

Before you ask why not the use the BME680 sensor component. The official Bosch BSEC library it’s a more sophisticated implementation made by the manufacturer.
@arcsur did awesome job porting the pimoroni python library to here. Unfortunately, Bosch doesn’t offer its library in python and also it’s not an open source to go with hass.

Here is what it does differently:

Uses the last days of measurements to auto-calibrate itself (last 4 or 28 days), getting more accurate with time
Calculates the IAQ (indoor air quality) and returns real IAQ values instead of the percentage
You can compensate for the high-temperature readings
At the moment there is no sea level pressure compensation (neither the component)
Support for 1.8v (closed cube) and 3.3v variants
It’s on C instead of python
It’s the official precompiled library made by the manufacturer and the one that is used in real products

More can be found here:
BSEC Integration Guide

The example here is for the raspberry pi.
There is also one for arduino (ESP32 and ESP8266) I can post here if you guys want.

Pre requisits:

Mosquitto Client
sudo apt-get install mosquitto-clients
i2c-tools (if you don’t know the i2c address of your BME680)
sudo apt-get install i2c-tools

How to:

Get the code:
git clone https://github.com/alexh-name/bsec_bme680_linux.git
Create a folder ./src inside the /bsec_bme680_linux
cd /bsec_bme680_linux
mkdir ./src
Download the Bosch Sensortech BSEC Library and unzip the files inside /bsec_bme680_linux/src do not change the name it should look like bsec_bme680_linux/src/BSEC_1.4.6.0_Generic_Release_20180425/
Open make.config with your favorite text editor and change the line #5 with you pi model
Pi Zero: ARCH="${VERSION}/RaspberryPI/PiZero_ArmV6-32bits"
RPi 3: ARCH="${VERSION}/RaspberryPI/PiThree_ArmV8-a-64bits"
Find out the i2c address of your sensor
i2cdetect -y 1
If is 77 change line #34 to int i2c_address = BME680_I2C_ADDR_SECONDARY;
No need to change if is 76

Now we need to change the output_ready function on the bsec_bme680.c to make it look like a json to MQTT. Change line #194-200 to:

printf("{\"IAQ_Accuracy\": \"%d\",\"IAQ\":\"%.2f\"", iaq_accuracy, iaq);
printf(",\"Temperature\": \"%.2f\",\"Humidity\": \"%.2f\",\"Pressure\": \"%.2f\"", temperature, humidity,pressure / 100);
printf(",\"Gas\": \"%.0f\"", gas);
printf(",\"Status\": \"%d\"}", bsec_status);

Now compile using ./make.sh
Test it using ./bsec_bme680. You should get some really low IAQ and Accuracy 0. It’s normal and it will progressively get more accurate.
To mqtt the result type the follow:
./bsec_bme680 | mosquitto_pub -h 192.168.1.XXX -u "your broker user" -P "your broker password" -p 1883 -t home/pizero/bme680 -l

Now it’s the annoying chore of using yaml. Hopefully, my example will help:

- platform: mqtt
  name: "BME680 IAQ Accuracy"
  state_topic: "home/pizero/bme680"
  value_template: "{{ value_json.IAQ_Accuracy }}"

- platform: mqtt
  name: "BME680 IAQ"
  state_topic: "home/pizero/bme680"
  value_template: "{{ value_json.IAQ }}"

- platform: mqtt
  name: "BME680 Temperature"
  state_topic: "home/pizero/bme680"
  value_template: "{{ value_json.Temperature }}"
  unit_of_measurement: "°C"
  device_class: temperature

- platform: mqtt
  name: "BME680 Humidity"
  state_topic: "home/pizero/bme680"
  value_template: "{{ value_json.Humidity }}"
  device_class: humidity
  unit_of_measurement: "%"

- platform: mqtt
  name: "BME680 Pressure"
  state_topic: "home/pizero/bme680"
  unit_of_measurement: 'hPa'
  value_template: "{{ value_json.Pressure }}"

- platform: mqtt
  name: "BME680 Gas"
  state_topic: "home/pizero/bme680"
  unit_of_measurement: 'G Ohms'
  value_template: "{{ value_json.Gas }}"

- platform: mqtt
  name: "BME680 Status"
  state_topic: "home/pizero/bme680"
  value_template: "{{ value_json.Status }}"

If you want to set the temperature offset edit bsec_bme680.c and change line #30.
#define temp_offset (0.5f)
Make sure that you recompiled again using step 6.

A special thanks for @skalavala and @gpbenton for the help with mqtt and alex-name for making a friendly C version.
If you have a Pimoroni UnicornHAT HD alex-name has a cool script for it.

This is not the most elegant solution but works. Let me know if you have ideas on how to improve it
Good luck

gpbenton · May 23, 2018, 4:13am

You could get the program to output messages for MQTT discovery to avoid having to add all that yaml.

ferazambuja · May 23, 2018, 4:26am

Great idea. I’m looking at the documentation and it looks like I would have to first print multiple consecutive configuration topic submissions.

rStorms · June 12, 2018, 4:48pm

Hi @ferazambuja,

upon running the compiled c file I get: stat’ing binary file: No such file or directory. Still the results are read and output and seem much more accurate than the original sensor. So far so good. Now, when I start the channel in the terminal via

sudo ./bsec_bme680 | mosquitto_pub -h localhost -p 1883 -t wohnzimmer-bme680 -l

how can I ensure that the script is running after I close the terminal. Have never worked with MQTT before

By the way, temperature in the original HA sensor is 5 degrees higher, than from the Bosch library…

Edit: Solved this with screen, not sure if this is the best way to go though

Thanks!

ferazambuja · June 12, 2018, 6:48pm

Me too, but disappears after a few days of use.

To run on boot I create a service for it. I can post here if you want.

rStorms · June 13, 2018, 6:16am

That would be awesome!

A few things I stumbled upon:

Values are shown with a very high frequency, making the graph not very helpful. Also the tremperature is fluctuating a lot between +/- 0.3 degrees, which adds to this problem. There are also a few peaks, where I have no idea where they come from. Is it the same for you? It would help to only get a moving average each 10 seconds or so.

The graph for the IAQ is only showing the state (like for lights) instead of a nice curve, as for the temperature. Any idea why this is? (Can’t show you a screen, as there are too many values in the chart, for HA to load them…) Also the IAQ seems to be very low with 250 for a clean German city, where I had the window open all night…

ferazambuja · June 13, 2018, 10:34pm

Yes, the sampling is high. I don’t think that there is much on the HA graph plotting that can be done, probably grafana can do a better job.
For reducing the frequency of measurements you would need to change the bsec_bme680.c code or have a script parsing averaging the data output before the mqtt. That would be great and not that complicated since is easy to get a csv output.
The way I’m doing right now is just sending what is “printed” on the screen to MQTT.

Add unit_of_measurement: "IAQ" on the yaml file for the IAQ.

I’m also getting low values, that’s the mystery of the BME680. Hopefully, more people will give it a try for us to compare.

ferazambuja · June 13, 2018, 10:51pm

Here is how to setup as service:

First the script to make it easier to run:
cd /bsec_bme680_linux
nano run.sh
Paste:

#!/bin/bash
/home/pi/bsec_bme680_linux/bsec_bme680 | mosquitto_pub -h 192.168.1.XXX -u "your broker user" -P "your broker password" -p 1883 -t home/pizero/bme680 -l

And save it.

Create the service:
sudo nano /etc/systemd/system/bsec_bme680.service

[Unit]
Description= Bosch BSEC BME680 Sensor
After=network.target

[Service]
WorkingDirectory=/home/pi/bsec_bme680_linux
ExecStart=/home/pi/bsec_bme680_linux/run.sh

[Install]
WantedBy=default.target

Configure
sudo chmod 744 /home/pi/bsec_bme680_linux/run.sh
sudo chmod 664 /etc/systemd/system/bsec_bme680.service
sudo systemctl daemon-reload
sudo systemctl start bsec_bme680.service
sudo systemctl status bsec_bme680.service and check if is working correctly
sudo systemctl enable bsec_bme680.service

You can find more info here:

Good luck

rStorms · June 15, 2018, 9:03pm

Thanks for that!

I wrote a little Python Script that goes into the BSEC root folder and directly runs the C script, then takes 20 values from the script and only publishes the median to MQTT. If you are interested I can post the script here…

digiblur · June 16, 2018, 12:55am

I use the BME280’s on ESP8266 chips and I comparing them to other sensors such as the DHT22 and such, I figured out quickly I was measuring too often and it was causing the chip to heat up. I wonder if the same can happen with the 680? I found a good balance to check the temperature every 60 seconds.

My BME280 & LDR :

ferazambuja · June 16, 2018, 2:15am

Please publish it here
I believe a python script is a better solution. I might even give a try to get MQTT discovery than we don’t need to type anything on the yaml file.

Could be the case, however, the BME680 is also running the heating element to get the gas measurements. Only testing two units side by side Some people even talk about surrounding components around the board can influence. Bosch talks about how to compensate for the surrounding heating.
From my experience these sensors are really good at tracking temperature changes, however, you need to compensate using another device.

What is good on the BME280 is that you don’t need the heating element on, unless you are running inside something and have condensation issues.

digiblur · June 16, 2018, 2:26am

Github link is above in previous post of course. I used this library:

I need to rename my sensor to my outdoor station as it’s a nodemcu now due to the better power regulator and voltage I’m feeding it. I’m going to check out their official library and modify my code to see how it does. Currently working well though! Great little sensor!

wemo3 wemos2 wemos1

rStorms · August 12, 2018, 3:09pm

Sorry for being so late. Busy times at work. I uploaded the file here: https://github.com/rstoermer/bsec_bme680_python/blob/master/bsec_bme680.py

ger_marsh · October 29, 2018, 9:35am

Thank you very much to all involved in this. The Python script does exactly what I wanted. However, when I try to use the commands to add it as a service, if fails after a reboot. I wonder if it is started after some dependency.

When I start the service interactively it works with this status:
pi@raspberrypi:~/bme680/examples/bsec_bme680_linux $ more test.lis ● bsec_bme680.service - Bosch BSEC BME680 Sensor
Loaded: loaded (/etc/systemd/system/bsec_bme680.service; enabled; vendor pres
et: enabled)
Active: active (running) since Sun 2018-10-28 18:19:26 GMT; 4s ago
Main PID: 18723 (run.sh)
CGroup: /system.slice/bsec_bme680.service
├─18723 /bin/bash /home/pi/bme680/examples/bsec_bme680_linux/run.sh
├─18724 python3 bsec_bme680.py
└─18731 ./bsec_bme680

I may put a extension from the Rpi to the card to move it from the CPU’s heat which is affecting the reading.

Thank you again.

azrael783 · November 2, 2018, 2:01pm

I think the high sampling rate is due to the mode the BME 680 is used in. The BME has three different modes:

continuous mode - sampling rate 1Hz --> for testing purposes
low power mode - sampling rate 0.33Hz
ultra low poer mode - sampling rate 3.3mHz

Unfortunately I’m not a programmer so I can’t tell you how you can change the mode of the BME. But maybe the manual of the BME can help.

scrubby18 · November 7, 2018, 1:42pm

Dear guys,
you have done all an awesome job when reading the threats from 2017 until today. Many thanks for the effort you have spend on it. Since a couple of days I’m looking into a solution to measure the air quality of various room in our house. In particular I wanted to distinguish if a room has not sufficient Oxygen (too much CO2) and/or if other unhealthy gases are in the air, e.g. formaldehyd. Since I have limited experiences with microcontrollers I’m looking for the combination Rasperry PI, ESP8266 or ESP32, Home Assistant, MQTT and then various sensors. I have now ordered one MH-Z19 and one BME680 sensor and when they arrive in a couple of weeks then I would like to implement it. I have not tried your manuals outlined above, because I do not have the sensors yet. However, do you know if anyone has a project page with some details about sticking together the hardware (wiring ESPs and sensors) etc. ? Then I would like to have some kind of reporting like was during the night too much CO2 in the room resulting in unhealthy sleep, or when is the maximum concentration of unhealthy gases in a room, etc. The background is that we have bought a very old house and we have detected already that a lot of different varnishes, glues, etc. have been used and some rooms may evapurate unhealthy substances.
So, what would I like to say with my post. I’m looking for some contacts for some help if I have my hardware together. Would it be possible to connect the MH-Z19 and the BME680 to one ESP? I’m not able yet to send personal mails. I hope that I do not bother you.
Many thanks and with best regards
Dirk

azrael783 · November 7, 2018, 3:11pm

Please be aware that the BME 680 is giving you only a so called “IAQ” value back. The sensor is not able to detect a special gas.

timothybrown · November 8, 2018, 9:54am

First off, I wanted to thank @rStorms for the Python script. A few weeks ago I started off just making one or two changes to it and cleaning a few things up. Before I knew it I had created an entire Python library that takes care of compiling the bsec_bme680 process, copying the config, creating the state file, running the actual bsec_bme680 process, decoding the JSON and wrapping it in an iterable generator.

Then I took that library and used it as the core of a daemon that basically does what @rStorms’ script did, only with more options. Major new features include:

Logging directly to the Systemd Journal. (If available.)
Runs as a first class Systemd daemon process with notify and watchdog support. (If available.)
Now supports Home Assistant’s MQTT Discovery feature!
No longer establishes a connection to your MQTT server every time it publishes a value, instead we use an async MQTT connection process, which will stay connected (and reconnect if needed) between publish events.
Switched to delivering the mean value instead of the median value. This results in smoother graphs in HA.
Added an option to increase the size of the sample cache. Basically, there’s two options of note here: update_rate and cache_multiplier. The former option controls how often we report data to MQTT, the latter option controls how much data we keep in the buffer to use for the averaging calculation. For example, let’s say you’ve got the BME680 set to sample every 3 seconds, you’ve got update_rate set to 60 and you’ve got cache_multiplier set to 5: The program will take a sample from the sensor every 3 seconds and save it in the buffer; every 60 seconds it will publish an average of the last (60/30=20)*5=100 Samples. If this sounds complex, don’t worry about it too much. I’ve gotten super smooth graphs, while still reacting quickly to changing conditions by using: update_rate=60, sensor_sample_rate=3 and cache_multiplier=5.
An option for reporting the IAQ value as a percentage (0% = Bad, 100% = Good).
Supports MQTT servers that require authorization.
Added an option to report temperature in F instead of C.
I modified the underlying bme680_bsec.c code to allow passing command line arguments to set options, instead of having to edit the #define variables in the source. This allows you to change BME680 options without it having to recompile the binary each time.
The above mentioned file has been embedded inside of the underlying BSECLibrary Python module, which handles all the compiling for you now anyway. (No more make file!)

Then, just to wrap it all up in a nice bow I created an installer script that takes care of downloading the Bosch BSEC package for you, extracting it, setting up a Systemd service, installing the needed Python dependancies (Debian based distros only) and telling you how to configure and use the service.

Basically all you have to do now is clone my Git repo, run install.py, set some options in the bsec-conduit.ini file and start the daemon with systemd start bsec-conduit. That’s it!

I call it BSEC Conduit and it’s available here: https://github.com/timothybrown/BSEC-Conduit

A bit overboard? Perhaps. But it only took me about a week of evening hacking to get it all together, and I wanted to put the underlying BSEC communication and compilation code into a dedicated Python library anyway, for use in other projects. Hopefully this will be useful to someone.

Please report any bugs, issues, problems, whatever to me, either here or on GitHub.

azrael783 · November 8, 2018, 1:34pm

Looks really great! Thank’s for your work. I will order a BME 680 and try it out. The integration in HA is still using MQTT right?

timothybrown · November 8, 2018, 1:59pm

Correct. I thought about doing direct HA integration for the next feature, however, I’m not sure it would ever be accepted as an official component due to the reliance on third-party code that has to be installed by hand (the BSEC library from Bosch).

That said, the way my BSECLibrary Python Module takes care of compiling the underlying binary, I could still make a functional HA component, it would just be non-official. You’d just have to copy it and the BSEC package into a special section of your HA config folder. (Versus it being installed automatically like official HA components do. Part of the requirements for those is that they’re available on the Python package index, which is what allows HA to auto-install the needed libraries when you enable a component in your config. Bosch’s license of the BSEC library doesn’t allow you to directly redistribute their source, just the resulting binaries. So, I’d have to pre-compile the binaries and distribute them with the module. That’s doable, but there are some other small catches as well.)

Anyway, MQTT seems to work well enough, and HA has a simple, built-in server if you don’t want to install Mosquitto just for this. This supports MQTT Discovery as well, so you don’t even have to edit your HA config.

Though, if you guys think making this into a component is worthwhile, I’ll do it. It’s only about an evenings worth of work (basically just a few import statements and some setup code).