Passive BLE Monitor integration (Xiaomi Mijia BLE MiBeacon monitor)

You’re welcome. AssertionError is fixed (thanks mcm69) in 0.6.9 (stable) and in 0.7.1(beta).

1 Like

Hey @TychoWerner!
Many moons ago, I struggled and managed to add a few sensors - my experience is here
Since I am an iOS user, I could not follow your exact steps the second time around I needed to add a few more but I found your instructions clear, concise and complete.

On another note, being a Tasmota fan, I tried using the HM 10 integration to add the fresh batch but I must say, this platform is superior and continues to be my preferred choice! Thanks @Magalex !

I finally did it, 6 Xiaomi LYWSD03MMC sensors working perfectly, with all the data, temperature, humidity and battery.

all the information in this link


Hi Carlos,

Sound interesting, I am considering to give it a try.

How quick are data updated in HA? In this current solution it is too slow and in some of my scenarios, where I need to act upon humidity reaching a maximum level, this won’t do, as the it takes too long time to get the value registered in HA.

And you don’t need the encryption bind_key neither?

Hi, so far it works fine
does not need encryption, and the update is at most every 5 minutes, I think that is fine,
even as it works well buy 4 more thermometers

Try to do this configuration, it worked for me, the other implementations were very complicated, with the 32-digit key, the MAC does not ask for anything here either

all success

Hi Carlos,

I have now spent several hours and got stuck at the point after running
ampy -p /dev/ttyUSB0 put ble.py and ampy-p /dev/ttyUSB0 put mqtt.py

After that, nothing new appears in the terminal.

it stops at

MicroPython v1.12 on 2019-12-20; ESP32 module (spiram) with ESP32
Type "help()" for more information.
>>>

Hope you can help with hints from your experience doing the installation.

I am using an esp32-Cam as mentioned in the guide should work

I am running the ampy commands from a Raspberry py as I didn’t succeed installing ampy on my Windows10.

I am uncertain about these points:

  • When I run the ampy commands, no output /result is given. it just waits for some seconds and finishes making ready for a new command in the terminal.
    o Is this the expected behavior?

  • I am not sure whether the board should be in the programming mode when running the ampy commands. I tried to do so then I get errors, so I think it shouldn’t

  • How do you figure out which USB port on the py you are attached to. Looks like it only accepts USB0 even though I move to another physical port!

  • Regarding mqtt

  • I assume it is necessary to turn: anonymous: true
    in the MQTT configuration in HAS as we don’t give the username and password in mqtt.py

  • and what about the port number i tried adding it to the IP address like this:
    mqtt_server = ‘192.168.0.100:1883’
    but still no new activity in the terminal

Thanks :blush:

Hello Ghassan
Exactly the same thing happened to me at that point, when modifying the file “mqtt.py”

  • The username and password of the Mqtt server must be entered in addition to the communication port
    the “main.py” file is loaded at the end of the process, once you have added the devices in the HA. The “main.py” file is used to make ESP32 run automatically when plugged in.

I attach the code of mqtt.py so you can see how to solve it
I hope it works

import time
from umqtt.simple import MQTTClient
import ubinascii
import machine
import micropython
import network
import esp
esp.osdebug(None)
import gc
gc.collect()
import ble, ntptime

ssid = 'x   wifi xxxxx'
password = 'xxx    wifi pass  xxxx'
mqtt_server = '192.168.0.35'
#EXAMPLE IP ADDRESS
#mqtt_server = '192.168.1.144'
client_id = ubinascii.hexlify(machine.unique_id())
topic_pub = b'espble'

def connect():
  global client_id, mqtt_server, topic_sub
  client = MQTTClient(client_id, mqtt_server, port=1883, user='xxxxxx', password='xxxxxxxxx')
  client.connect()
  print('Connected to {} MQTT broker'.format(mqtt_server))
  return client

def restart_and_reconnect():
  print('Failed to connect to MQTT broker. Reconnecting...')
  time.sleep(10)
  machine.reset()

station = network.WLAN(network.STA_IF)

station.active(True)
station.connect(ssid, password)

while station.isconnected() == False:
  pass

print('Connection successful')
print(station.ifconfig())

try:
    ntptime.settime()
    print (ble.timestamp())
except Exception as e:
    ble.debug('Error: ntptime settime ' + str(e))

try:
  client = connect()
except OSError as e:
  restart_and_reconnect()

myBLE = ble.ble()
myBLE.setup()

print ('Found:')
for a in myBLE.addresses:
    type, address, name = a
    ble.debug ('Found Address: {} Name: {}'.format(ble.prettify(address),name))

lastday = 0
while True:
    # update the RTC once a day
    today = ble.timestamp('day')
    if today != lastday:
        try:
            ntptime.settime()
            lastday = today
            ble.debug('Time set from server')
        except Exception as e:
            ble.debug('Error: ntptime ' + str(e))

    # cycle through the captured addresses
    for a in myBLE.addresses:       
        type, myBLE.address, name = a
        # if this is a LYWSD03MMC'
        if name == 'LYWSD03MMC':
            # if we are successful reading the values
            print ('\r\n----------------------------------------------------------')
            if(myBLE.get_reading()):
                message = '{"temperature": "' + str(myBLE.temp) + '", '
                message = message + '"humidity": "' + str(myBLE.humidity) + '", '
                message = message + '"batt": "' + str(myBLE.batteryLevel) + '", '
                message = message + '"voltage": "' + str(myBLE.voltage) + '"}'
                print (message)
                topic = topic_pub + '/' + ''.join('{:2x}'.format(b) for b in myBLE.address)
                print (topic)
                try:
                    client.publish(topic, message)
                except Exception as e:
                    ble.debug('Publish ' + str(e))
                    try:
                        client.disconnect()
                        client = connect()
                    except OSError as e:
                        restart_and_reconnect()

    # wait a minute for the next one
    time.sleep(60)

Hello I have now managed to integrate the sensors into my homebridge and read out the bidkey.
So far the sensors have been running but now I get the message for each sensor.
Does anyone know this message or can you tell me how to get this error fixed?
It looks to me as if the sensors are not connected via Bluetooth.

Hi Carlos,

Thanks for your input :blush:

I added the port, username, and password to the MQTTClient command as in your case but this did not give any visible change.

Still nothing happens in the terminal.

don’t we need anything to trigger starting the MQTT connect procedure?

Do you send any commands through terminal or is it supposed to start by itself after reset?

I think we are missing something basic. Hope you have a clue :blush:

Thanks.

Hello Ghassan
I am not very expert on the subject, it also cost me a lot to get it to work.
Maybe it can help you to use the Thonny Python IDE program
there in options you select the type of board you are using, you can also load the fimware from there
The file “main.py” is loaded at the end, once you see that it is receiving data in HA. The main.py makes the ESP run automatically in the event of a restart.

I hope you find it
stay tuned

Hello
query
How can I put the time of the last reading on the card?
I want to see the same data that appears when you click on the sensor but on the dashboard card

Thank you

Captura de Pantalla 2020-07-13 a la(s) 20.59.09

Hello Carlos,

Good news :slight_smile:

As I suspected, I was missing something that triggers/starts the MQTT Connect procedure.

And I discovered that this is what happens from the main.py

So, I just uploaded main.py and everything worked as expected,

I have 6 devices and I get reading from all of these. I just think I have an issue with the Bluetooth range as sometimes the devices that are most far away from my esp32 fails to load. Maybe this can be solved with a full size esp32 board.

It Is great that this alternative way of connecting to the Xiaomi passive BLE Monitor sensor works too.

Unfortunately, the guide is not precise enough

Thanks for your support :blush:

Hello Ghassan
I’m glad it works
Regarding the range, I have seen in some pages that modify the ESP 32 antenna and solder a cable with an external antenna, I have also seen that there are plates with the external antenna connector, but it is always good to find the best position in the house where you can capture all the signals and have no problems.

I have been testing the board for a few days and I have realized that at least once a day I have to restart the board because it is not sending a signal, … if you find a solution to that, tell me

regards

Hello Carlos.

My first impression of the esp32 solution is that it might work but due to the none complete and in some cases misleading guide and not having the possibility to directly ask the developer, my expectations are not that high but I will test it for a while besides the one based on the custom component which is the main topic of this discussion thread.

The only thing I am missing is a quicker update in HA. maybe there is a way. but otherwise it is very stable, and Aleksey, the developer of this custom component, actively participate in the discussions and reacts quickly on issues and questions.

I will encourage you to try it and I will be very glad to help you through😊

Hi @Magalex ,

I am running your custom component and I am glad for that. thanks for the good work.

I have 6 LYWSD03MMC sensors and they are updating about each 10-11 min.

I think this is too long especially if I need to do automation and react on for example high humidity level in the bathroom, while taking a shower, and start the ventilation.

Is it the expected behavior or can we make it update more often?

I am running the beta version 0.7.1, with this configuration:
´

  - platform: mitemp_bt
    rounding: false
    decimals: 1
    period: 60
    log_spikes: False
    use_median: False
    active_scan: false
    hci_interface: 0
    batt_entities: False
    encryptors:
                #temp1
               'A4:C1:38:EA:9D:B2': 'EAF04C41C87FC38B9B8948EC8F33A98C'
                #temp2
               'A4:C1:38:D7:19:FD': 'E3AA76D19C03047F4B13867BE5A9F627'
                #temp3
               'A4:C1:38:30:9E:02': 'F2C5DFB5AEA86802BA36822476C5864C'
                #temp4
               'A4:C1:38:70:6E:C1': 'E385A76844068AAE4A5897C643C688AF'
                #temp5
               'A4:C1:38:CB:26:E5': '0AEF0C3D2D0A5AAF3CCE2CBEE10DF357'
                #temp6
               'A4:C1:38:14:14:6E': '7C55FA64E547C0A3B8CDEF477884F096'
    report_unknown: false

thanks :slight_smile:

This is the expected behaviour, as the component from Magalex is only scanning the signals your sensors send (BLE). Not like the built-in component that sends out signals.

The reason for this is to provide a significant longer battery life. And in theory the sensor should send out a new signal, as soon as an update is registered (in the sensor). I just checked, I sometimes get no new signals for two or more hours, but as soon as something changes, the sensor sends a new signal. :slight_smile:

Hi Patrick,

Thanks for your response.

My experience is different. Even though measured and displayed values cahnge on the sensor, It takes in some cases more than 10 minutes before HA is updated with the same values.

This can’t be right.

I hope Aleksey can clarify

That doesn’t sound right, especially as I’m not having this delay. :laughing: :laughing:

Hi @Magalex ,

I am running the beta version 0.7.1 of the mitemp_bt custom component

My LYWSD03MMC devices firmware was upgraded recently to: 0109

For simplicity I run with only one device, tried to vary the period and , active scan but this did not give any visible difference on how often the readings are updated in HA.

Here is my configuration:

- platform: mitemp_bt
rounding: false
decimals: 1
period: 15
log_spikes: False
use_median: False
active_scan: true
hci_interface: 0
batt_entities: False
encryptors:
   'A4:C1:38:D7:19:FD': 'E3AA76D19C03047F4B13867BE5A9F627'
report_unknown: false
whitelist: true

I have done some debugging by adding various logging in sensor.py.

Here is an extract from the log:

2020-07-16 18:33:06 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: data from mac:b'\xfd\x19\xd78\xc1\xa4' packet: 79 
2020-07-16 18:33:06 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: Humidity collected (A4C138D719FD): 57.6 
2020-07-16 18:33:18 ERROR (SyncWorker_4) [custom_components.mitemp_bt.sensor] GHC:Update start, Entity name:mi h_A4C138D719FD 
2020-07-16 18:34:07 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: data from mac:b'\xfd\x19\xd78\xc1\xa4' packet: 81 
2020-07-16 18:34:07 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: temperature collected (A4C138D719FD): 24.7 
2020-07-16 18:34:20 ERROR (SyncWorker_4) [custom_components.mitemp_bt.sensor] GHC:Update start, Entity name:mi t_A4C138D719FD 
2020-07-16 18:44:05 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: data from mac:b'\xfd\x19\xd78\xc1\xa4' packet: 83 
2020-07-16 18:44:05 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: Humidity collected (A4C138D719FD): 76.7 
2020-07-16 18:44:11 ERROR (SyncWorker_18) [custom_components.mitemp_bt.sensor] GHC:Update start, Entity name:mi h_A4C138D719FD
2020-07-16 18:45:06 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: data from mac:b'\xfd\x19\xd78\xc1\xa4' packet: 85 
2020-07-16 18:45:06 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: temperature collected (A4C138D719FD): 23.5 
2020-07-16 18:45:12 ERROR (SyncWorker_7) [custom_components.mitemp_bt.sensor] GHC:Update start, Entity name:mi t_A4C138D719FD 
2020-07-16 18:55:06 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: data from mac:b'\xfd\x19\xd78\xc1\xa4' packet: 87 
2020-07-16 18:55:06 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: Humidity collected (A4C138D719FD): 66.8 
2020-07-16 18:55:16 ERROR (SyncWorker_13) [custom_components.mitemp_bt.sensor] GHC:Update start, Entity name:mi h_A4C138D719FD 
2020-07-16 18:56:05 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: data from mac:b'\xfd\x19\xd78\xc1\xa4' packet: 89 
2020-07-16 18:56:05 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: temperature collected (A4C138D719FD): 22.8 
2020-07-16 18:56:17 ERROR (SyncWorker_8) [custom_components.mitemp_bt.sensor] GHC:Update start, Entity name:mi t_A4C138D719FD 
2020-07-16 19:06:05 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: data from mac:b'\xfd\x19\xd78\xc1\xa4' packet: 91 
2020-07-16 19:06:05 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: Humidity collected (A4C138D719FD): 61.9 
2020-07-16 19:06:07 ERROR (SyncWorker_16) [custom_components.mitemp_bt.sensor] GHC:Update start, Entity name:mi h_A4C138D719FD 
2020-07-16 19:07:04 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: data from mac:b'\xfd\x19\xd78\xc1\xa4' packet: 93 
2020-07-16 19:07:04 ERROR (SyncWorker_11) [custom_components.mitemp_bt.sensor] GHC: temperature collected (A4C138D719FD): 22.9 
2020-07-16 19:07:08 ERROR (SyncWorker_7) [custom_components.mitemp_bt.sensor] GHC:Update start, Entity name:mi t_A4C138D719FD 

And my observations can be summarized as follows:

  • The device sends humidity and temperature in sperate messages with about 1 min between these. Then waits 10 more minutes and send the humidity and temperature again wait 10 min and so on.
  • The same package is transmitted several times (I removed logging this)
  • We only get every second package.

But comparing to the behavior in the Xiaomi Home app, data are updated several times per minute!

  • Is it because the devices recognize the app and updates it more frequently?
  • Is it because the mitemp_bt component connects to the device in another way?
  • And can this be changed to get more updates

For comparison, I have also installed the esp32 based component Xiaomi_LYWSD03MMC_for_HA developed by Anthony Norman and it does not use encryption keys to connect to the devices:

here are some lines from the com port monitor where I only included the same device as above:

[20:41:08]{"temperature": "25.46", "humidity": "51", "batt": "76", "voltage": "2.858"}
[20:41:08]b'espble/a4c138d719fd'
[20:42:34]{"temperature": "27.16", "humidity": "64", "batt": "76", "voltage": "2.858"}
[20:42:34]b'espble/a4c138d719fd'
[20:44:41]{"temperature": "30.97", "humidity": "85", "batt": "76", "voltage": "2.858"}
[20:44:41]b'espble/a4c138d719fd'
[20:46:06]{"temperature": "32.61", "humidity": "83", "batt": "76", "voltage": "2.858"}
[20:46:06]b'espble/a4c138d719fd'
[20:47:42]{"temperature": "33.72", "humidity": "71", "batt": "76", "voltage": "2.858"}
[20:47:42]b'espble/a4c138d719fd'
[20:49:16]{"temperature": "34.4", "humidity": "66", "batt": "77", "voltage": "2.87"}
[20:49:16]b'espble/a4c138d719fd'

As we can see here, the updates are much more often than with mitemp_bt

But I find mitemp_bt more stable and mature.

Aleksey I hope you can be inspired by Anthonys solution, and probably join forces and gather the best from both components. It will be great if you can get more often updates which increases usage cases, and if we get rid of the need for the encryption keys the solution becomes much easier to setup.

Aleksey please let me know what you think.

Kind regards,
Ghassan

Hi Patrick,

What firmware version is running on your Xiaomi devices?
You can check that in the Xioami Home app…