Passive BLE Monitor integration (Xiaomi Mijia BLE MiBeacon monitor)

Hi,
it’s a good news! I can’t find on your github repo the advertising payload you refer to.
Do you have any link?

Thanks for the great Work! LYWSD03MMC with custom fw…soon a Sensor in ervery rooms.

I’ve added a first draft explanation of reverse engineering the MiBeacon protocol on our repository.

1 Like

Which version/build of “Python” works correctly with this component under Windows 10 ?

Hi all,

I’m still trying to run this component (ble_monitor) under Home Assistant for Windows Portable https://github.com/AlexxIT/HassWP (v0.116.4 - 2020-10-22, Python v3.8.5 32bit)

I just want to see if I can make my Xiaomi “Temperature and Humidity Monitor 2” LYWSD03MMC sensors working on a fresh installation of Home Assistant with Bluetooh Low Energy (BLE) component under Windows 10.
I have already tried 7 times (install and re-install, numerous tries…).

If it’s definetly not possible with Home Assistant under Windows 10 , just let me know… (I will stop tries).

Like everybody, I get stuck with the famous error :

"\Home Assistant\config\custom_components\ble_monitor\aioblescan_ext.py", line 1383, in create_bt_socket
    sock = socket.socket(family=socket.AF_BLUETOOTH,
**AttributeError: module 'socket' has no attribute 'AF_BLUETOOTH'**

That’s almost crazy : everybody says "rebuild your Python 3.x.x, make sure you have included “Bluetooth.h”, try reinstall BlueZ… But nothing works (on Windows 10, Home Assistant in December, 24) !

I know I have to rebuild a totally new Python v3.8.x or v3.9.x (I will see) on my Windows system, but I didn’t know I had to do this before using this very ‘formidable’ app Home Assistant on a my system (just Windows 10, the most used OS in the world !)…

Numerous Issues and messages have been posted on the subject, but still no clear solution for (very lonely) Windows users…

Should I have to investigate a rebuild/compilation of Python under my Windows 10 system, just to see my simple temperature sensor work on Home Assistant ?
Really ?

I’m open to all working solutions, now… (including dev if it’s really necessary, and including flashing my Xiaomi sensors (https://github.com/atc1441/ATC_MiThermometer) !)

Please, help !

Domi

PS: If I can get this Xiaomi sensor working LYWSD03MMC under Home Assistant for Windows, I promise you to publish a very nice tutorial !

PS2 : The goal is very simple : PC, Windows 10, Home Assistant, Xiaomi temp + humid sensor ref LYWSD03MMC. Is it even possible with Home Assistant ?

Has anyone tested out the new intergration for the MCCGQ02HL Door/window sensor?

Thanks for that. I will test with fake data on sunday maybe.

someone tested with supervised install on debian ? i got no sensors data, but got model and sensors added.

2020-12-27 13:27:17 DEBUG (MainThread) [custom_components.ble_monitor] Initializing BLE Monitor entry (config entry): <homeassistant.config_entries.ConfigEntry object at 0xffffa3a6aa40>
2020-12-27 13:27:17 DEBUG (MainThread) [custom_components.ble_monitor] async_setup_entry: domain {}
2020-12-27 13:27:17 DEBUG (MainThread) [custom_components.ble_monitor] async_setup_entry: {'hci_interface': [0], 'period': 60, 'discovery': True, 'active_scan': True, 'report_unknown': False, 'batt_entities': False, 'rounding': True, 'decimals': 1, 'log_spikes': False, 'use_median': False, 'restore_state': False, 'devices': [{'mac': '58:2D:34:52:FD:44', 'name': '582D3452FD44', 'temperature_unit': '°C'}], 'is_flow': True}
2020-12-27 13:27:17 DEBUG (MainThread) [custom_components.ble_monitor] HCI interface is [0]
2020-12-27 13:27:17 DEBUG (MainThread) [custom_components.ble_monitor] Spawning HCIdump thread
2020-12-27 13:27:17 DEBUG (MainThread) [custom_components.ble_monitor] HCIdump thread: Init
2020-12-27 13:27:17 DEBUG (MainThread) [custom_components.ble_monitor] 0 encryptors mac:key pairs loaded.
2020-12-27 13:27:17 DEBUG (MainThread) [custom_components.ble_monitor] whitelist: []
2020-12-27 13:27:17 DEBUG (MainThread) [custom_components.ble_monitor] 0 whitelist item(s) loaded.
2020-12-27 13:27:17 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: Run
2020-12-27 13:27:17 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: connected to hci0
2020-12-27 13:27:17 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: start main event_loop
2020-12-27 13:27:21 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Starting binary sensor entry startup
2020-12-27 13:27:21 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] BLE binary sensors updater initialization
2020-12-27 13:27:21 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] BLE binary sensors updater initialized
2020-12-27 13:27:21 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Binary sensor entry setup finished
2020-12-27 13:27:21 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Binary entities updater loop started!
2020-12-27 13:27:22 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Starting measuring sensor entry startup
2020-12-27 13:27:22 DEBUG (MainThread) [custom_components.ble_monitor.sensor] BLE sensors updater initialization
2020-12-27 13:27:22 DEBUG (MainThread) [custom_components.ble_monitor.sensor] BLE sensors updater initialized
2020-12-27 13:27:22 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Measuring sensor entry setup finished
2020-12-27 13:27:24 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Entities updater loop started!
2020-12-27 13:27:25 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Temperature sensor with mac address 58:2D:34:52:FD:44 is set to receive data in °C
2020-12-27 13:27:26 DEBUG (MainThread) [custom_components.ble_monitor.sensor] async_added_to_hass called for ble temperature 582D3452FD44
2020-12-27 13:27:26 DEBUG (MainThread) [custom_components.ble_monitor.sensor] async_added_to_hass called for ble humidity 582D3452FD44
2020-12-27 13:28:21 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] 1 MiBeacon BLE ADV messages processed for 0 binary sensor device(s) total. Priority queue = 0
2020-12-27 13:28:24 DEBUG (MainThread) [custom_components.ble_monitor.sensor] 2 MiBeacon BLE ADV messages processed for 1 measuring device(s).
2020-12-27 13:28:24 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: main event_loop stopped, finishing
2020-12-27 13:28:24 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: Scanning will be restarted
2020-12-27 13:28:24 DEBUG (Thread-5) [custom_components.ble_monitor] 473 HCI events processed for previous period.
2020-12-27 13:28:24 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: Run
2020-12-27 13:28:24 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: connected to hci0
2020-12-27 13:28:24 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: start main event_loop
2020-12-27 13:29:21 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] 2 MiBeacon BLE ADV messages processed for 0 binary sensor device(s) total. Priority queue = 0
2020-12-27 13:29:24 DEBUG (MainThread) [custom_components.ble_monitor.sensor] 1 MiBeacon BLE ADV messages processed for 1 measuring device(s).
2020-12-27 13:29:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: main event_loop stopped, finishing
2020-12-27 13:29:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: Scanning will be restarted
2020-12-27 13:29:25 DEBUG (Thread-5) [custom_components.ble_monitor] 453 HCI events processed for previous period.
2020-12-27 13:29:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: Run
2020-12-27 13:29:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: connected to hci0
2020-12-27 13:29:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: start main event_loop
2020-12-27 13:30:22 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] 0 MiBeacon BLE ADV messages processed for 0 binary sensor device(s) total. Priority queue = 0
2020-12-27 13:30:25 DEBUG (MainThread) [custom_components.ble_monitor.sensor] 0 MiBeacon BLE ADV messages processed for 1 measuring device(s).
2020-12-27 13:30:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: main event_loop stopped, finishing
2020-12-27 13:30:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: Scanning will be restarted
2020-12-27 13:30:25 DEBUG (Thread-5) [custom_components.ble_monitor] 482 HCI events processed for previous period.
2020-12-27 13:30:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: Run
2020-12-27 13:30:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: connected to hci0
2020-12-27 13:30:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: start main event_loop
2020-12-27 13:31:22 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] 1 MiBeacon BLE ADV messages processed for 0 binary sensor device(s) total. Priority queue = 0
2020-12-27 13:31:25 DEBUG (MainThread) [custom_components.ble_monitor.sensor] 1 MiBeacon BLE ADV messages processed for 1 measuring device(s).
2020-12-27 13:31:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: main event_loop stopped, finishing
2020-12-27 13:31:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: Scanning will be restarted
2020-12-27 13:31:25 DEBUG (Thread-5) [custom_components.ble_monitor] 464 HCI events processed for previous period.
2020-12-27 13:31:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: Run
2020-12-27 13:31:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: connected to hci0
2020-12-27 13:31:25 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: start main event_loop

removed component and restarted hass then added it agian. got data !

2020-12-27 13:37:12 DEBUG (MainThread) [custom_components.ble_monitor] HCI interface is [0]
2020-12-27 13:37:12 DEBUG (MainThread) [custom_components.ble_monitor] Spawning HCIdump thread
2020-12-27 13:37:12 DEBUG (MainThread) [custom_components.ble_monitor] HCIdump thread: Init
2020-12-27 13:37:12 DEBUG (MainThread) [custom_components.ble_monitor] 1 encryptors mac:key pairs loaded.
2020-12-27 13:37:12 DEBUG (MainThread) [custom_components.ble_monitor] whitelist: []
2020-12-27 13:37:12 DEBUG (MainThread) [custom_components.ble_monitor] 0 whitelist item(s) loaded.
2020-12-27 13:37:12 DEBUG (Thread-111) [custom_components.ble_monitor] HCIdump thread: Run
2020-12-27 13:37:12 DEBUG (Thread-111) [custom_components.ble_monitor] HCIdump thread: connected to hci0
2020-12-27 13:37:12 DEBUG (Thread-111) [custom_components.ble_monitor] HCIdump thread: start main event_loop

looks like problem was with encryptors mac:key pair
case closed

Hi @Magalex, firstly I’d like to say thank you for putting this integration together. I read above that you only receive complaints, but I feel it’s just people using your integration and having issues, or questions, just as I’m about to post here. So although I have a question, I’m very appreciative for people like yourself who go through the trouble of developing these products for the community.

So here’s my issue: I had an RP4, with an SD card, a BT dongle (https://www.amazon.com/gp/product/B002AQSTXM/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1) and 4 Xiaomi Mijia Temp and Humidity Sensors LYWSDCGQ spread out throughout the house (basement and upper floor).

I had the following in my configuration.yaml:

mitemp_bt:
  rounding: True
  decimals: 1
  active_scan: False
  batt_entities: True
  hci_interface: 
                - 0
                - 1

My setup was working perfectly.

Once you updated the repo, I also updated the yaml to be:

ble_monitor:
  rounding: True
  decimals: 1
  active_scan: False
  batt_entities: True
  hci_interface: 
                - 0
                - 1

And it worked.

I then replaced the SD with an SSD, and 3 of my 4 sensors stopped working.

I’ve tried everything I know (I’m not an expert), with no luck, even backing out everything and trying again:

  1. Uninstalled the integration
  2. Uninstalled the repo in HACS, commented the yaml code and restarted
  3. Factory reset the sensors (by holding the BT button on them until it starts flashing for about a minute)
  4. Re-paired them to the app on my phone
  5. Installed the repo in HACS and restarted
  6. Installed the integration and used the integration to setup (not yaml)
  7. But nada, I don’t see the devices
  • My sensors are broadcasting since I can see the values in the app.
  • It’s not a coverage issue since it was working perfectly before
  • I read that SSD could cause interference, but it worked for one sensor after I switched, which is odd. I put the SSD about a foot away from the RP4, not sure if that’s enough distance to help with the possible interference.

I’m not entirely sure how else to debug this, or what could be wrong.

I’d appreciate any support you can provide.

Thank you again for all your work, and happy new year!

@petermcandrew I have this sensor. I will answer any questions.

@celoberger Thanks for the kind words. I tend to think that this is still an EMC problem. And it’s not just radio interference - the purity of the power supply is of great importance in this area, the interference spreads not only over the air, but also along the conductors (power supply, in the first place). In desktop computers and server platforms, this issue is given more attention than in the case of RPi (separate power lines, high-quality filtering).

Have you tried bringing “failed” sensors closer to the host?

First of all, I advise (if you have not tried it yet) to take out the dongles on usb extension cords, and try adding ferrite rings to them. In addition, I think it is worth paying attention to such an aspect as sufficient power supply.

Tell me more about how your SSD is connected. This device has noticeably increased consumption, and it’s worth making sure that your power adapter and power wires are capable of transferring enough power to the raspberry. Many are skeptical when this issue is raised in the context of problems with raspberries, but I can say with complete confidence that the stability of the operation of the elements of the raspberry and its periphery really depends on this, and I remember several cases when the transition to better wires and power supplies solved some strange problems.

Hi,

First of all, thank you for this integration, it worked great for me. I had 3 LYWSD03MMC sensors that I had paired with Mi Home app in Europe, then bought another 4 from China and didn’t link them to Mi Home but rather added them directly to HA.

All of them were working fine until I upgraded BLE to the latest version . I tried first adding them manually, then I added them all to Mi Home app and tried again. Reinstalled BLE, cleared all devices, tried everything. They show up as devices but their state remains in status Unknown.

I didn’t add them manually using YAML, the encryption key is being read directly and I don’t see any errors in the log.

Anything obvious I am missing ?

Hard to tell what is wrong. Did you enable debug logging? Did you double check the bind key, they change upon adding the device to Mi Home. Perhaps it better to open an issue on github.

Just to follow up regarding the Brifit sensor: https://www.amazon.de/-/en/gp/product/B085C963Y3/ref=ppx_yo_dt_b_asin_title_o06_s00?ie=UTF8&th=1
Works pretty well with this custom component:
https://github.com/araines/homeassistant-thermoplus-ble.
But wouldn’t it make sense to merge the two? Seems to me they are pretty much doing the same thing. I donno.

I have the exact same experience.
Pretty much unable to run RPi 4 on SSD and at the same time use a bluetooth USB dongle reliably. Let alone 3 bluetooth USB dongles. I have tried with the official RPi 4 power supply. And with 1 meter USB extension cables. Don’t make much difference. I am going to try with some EMF shielding tape/cover of some sort in the near future.

We will need some more info to see if we can support this. Please open an issue on github and show the info from the following command

btmon --write hcitrace.snoop | tee hcitrace.txt

Copy the relevant parts of your sensor to see if it follows the Mi-Beacon format (more or less)

1 Like

I reinstalled the component, used Telia flasher to get new keys, it seems to be working now. It probably had an issue with the keys, they were cached somehow with the values before binding them to Mi Home. Thank you again, I got used to seeing the sensors, I was so lost without that :smiley:

Hello guys.

First of all, many thanks to author for this great integration. Works like a charm and already made my life easier.

I have quite basic question. I use LYWSD03MMC xiaomi sensor in my automation and couldn’t figure out something:

  • in order to make this integration working I need to get an encryption key or install a custom firmware for the Xiaomi Thermometer.
  • both ways work great, I can add the device to HA and poll the data. But at the same time, device becomes unreachable in my Apple iphone Xiaomi Home app.

From my understanding when I click “Activate” on atc1441 website in order to get a key, the sensor is being disconnected from Xiaomi cloud. If I remove the device in the app and add it back again, it can work in the xiaomi application, but doesn’t work in HA (no data, because of a new key?).
If I flash a sensor with the custom firmware, everything works great again in HA, but I can’t add the device in Xiaomi app at all (error 102 while adding a sensor).

Is there any way to use the sensors (get the data) in HA and Xiaomi app at the same time?
I’m going to workaround this problem with Xiaomi gateway, but this will not be straightforward.

… found the info in this thread… will try

Yes, if you want to use with both Hassio and Mi Home app, then you need to use the bind key generated by the Mi App.
You can get it by sniffing the network traffic, example by Stream app, export the .har file, send it to a computer, open the file in Chrome IE or other network browser and search for bind_key

Would it be possible to provide more details how to get this bind_key now?
Can you maybe provide your script ?

Hello, First of all, thank you for you great work!,.

I’ve integrated several LYWSD03MMC xiaomi sensors by activating it on reported atc1441 website.

My answer is … Could the sensors be added by other users to their own xiaomi account… This way HA wouldn’t read data sensor because key would be changed… Could this occur?.
Thanks in advance!.

@Carlos-HA and @Lexa22

As explained by @Carlos-HA in his edit, every time you ‘activate’ the sensor (on the ATC1441 website or in the Mi-Home app) it will generate a new bind-key (= encryption key). If you want to use both the mi-home app and use it with our custom component, you will need the bind-key when activating it on Mi-Home. To do this, you will have to “sniff” the communication between the app and sensor at the moment it is activated in the app. Our FAQ gives 3 options:

  1. ATC website
  2. MiHome mod
  3. Sniffing the comunication

Option 1 is not possible, as it won’t be connected to the Mi-Home app at the same time.

Option 2 is also not possible, after a second thought, as it will be connected to MiHome mod, and not to the official Mi Home. However, if you are willing to continue using Mi-Home mod in stead of the official Mi-Home app, you can use this (link). (note, I’m not sure if you can update from Mi-home mod to Mi-home without reactiving the sensors)

Option 3 has become more difficult recently, due to the fact that Xiaomi has enabled additional encryption of API requests recently. We don’t have a good description of the entire process, but @Magalex has made a start in this issue. So, option 3 is the best option if you want to use Mi-Home, but we need someone to write up a good step by step explanation.

A last note, sensors that are flashed with the ATC advertising format won’t need an encryption key, but they also won’t work in the Mi-Home app. This is because the BLE advertisements are in a different format. I’m not sure if Mi-Home understands sensors that are flashed with custom firmware that advertise in the mi-like format

1 Like

Hi Ernst.

Thanks for this summary message. This case (simultaneous use in HA and Mi app) was not clear enough for me after reading the FAQ. Now everything works fine in my scenario and both ways (modified android app and sniffing the traffic) helped me to get the encryption key without modification.
Again, thanks for your work!