Passive BLE Monitor integration

@La-te

This is how it is “designed” at the moment. Motion and acceleration should be reported instantly, otherwise, motion detection would not work (would take 60 seconds before motion is detected, if you move in second 1 of the period).

Oh yeah true it needs to be that way for the motion detection. Thanks for your quick response!

Anyway, I would like to limit the amount of data stored to the Home Assistant database for those motion sensors. I suppose now all those motion events from RuuviTag are recorded. Is there a sensible way to limit the amount of motion events recorded within this integration or Home Assistant overall? I would like to have at least this 10 days of history for all Home Assistant events and the motion data. However, for motion I suppose it would be enough to just record some “motion start and end” or “was moving during last minute” rather than each message from a RuuviTag. I have no idea if that is feasible.

The current and upcoming possibilities I can come up with but they might not be optimal today

  • Play something with the Home Assistant recorder settings. At least it would be possible to prevent motion events recorded at all but still have the live data. However, In my current use case I’m more interested in the statistics than the live data. I’m following when the door was opened. With this approach I could still have the other history data and throw away only the motion data.
  • The latest Home Assistant core 2021.8.0 seems to be bringing lots of help for the performance issue of history, statistics and graphs through it’s long-term statistics feature. However, I understood that each integration needs to be first updated in order for them to record the long-term statistics on top of the events. It might not be reality yet today to rely only on long-term statistics for all of my statistics and then limit the event database history to less than 10 days, like 2 days only if the performance is fine then. Is Passive BLE Monitor supporting long-term statistics (already or soon)? This might be still the best option for longer term once all integrations utilise long-term statistics. If motion events can be stored as long-term statistics then I could pretty safely just disable recording them in the recorder and have the long-term statistics only and still 10 days of event history for all the other data.
  • Install beta firmware to RuuviTag and configure it to send data less frequently
  • Move Home Assistant database to some more powerful platform than Raspberry Pi 4 SD card. Based on some other posts on the history graph performance issues, the whole thing is solved if you store the data on faster filesystem like on a remote database on some server with a disk. However, for simplicity I would prefer all on the same Raspberry Pi 4. It is better to rely on one fragile component than two fragile components with some custom implementation.

Is there any way to set bluetooth device according to the HCI number through GUI configuration?
(rather than MAC address)

I’ve got 2 bluetooth dongles. One for this integration, the other for bluetooth device tracking and controlling other BT devices. The ageing built-in BT tracking integration only selects based on HCI number. But a few times after host machine restart, the HCI number of the devices get swapped around, meaning BT integrations clashes and both kept failing until I changed MAC address manually.

No, only with YAML.

BLE monitor is, in the background, still based on the HCI number. On startup/initiation, BLE monitor will get a list of HCI numbers and corresponding MACs. After you select a MAC, it will look up the corresponding MAC and will use that. If the HCI numbers change after that, it will continue to use the HCI number it had in the beginning, so it will probably start to use the other BT interface.

Isn’t it possible to fix the HCI number to a MAC on OS level?

I will look into the possibility to reduce the amount of stored datapoints for these types, but I’m not sure if this is possible.

1 Like

May I ask how might one fix HCI number at OS level?

I’m running HA OS on proxmox host. So I guess I need to set it twice. But I believe HA OS uses a different bluetooth tool to Debian. (not hcitool)

To be honest, I don’t know. But if you were able to set the HCI number in the GUI, how would this solve your problem if the HCI numbers swap? The advantage of using the MAC address is that it will look up the HCI number, based on the MAC, which does not change.

The built-in BT device tracker integration works has a HCI number setting. I could set this BLE monitor to a different HCI number, then they should never clash.

good day,

pretty new to home assistant, and possibly using the passive ble integration is a high bar for my first go-around.

trying to integrate a govee 5051, and appear to have it being recognized. however, no attributes are passed along with the update (looking in developer tools for the output). is this normal ? -

device_tracker.ble_tracker_e3605921e492

source_type: bluetooth_le
mac: E3:60:59:21:E4:92
last seen: 2021-08-11T20:23:11.294482+00:00
rssi: -66
friendly_name: ble tracker E3605921E492

i figured the temp/humidity values would be passed along as attributes with this output, unless what i’m seeing is not a confirmation that the device is actually being polled.

any pointers much appreciated,

  • dsm

Did you enable active_scan?. Govee needs this option to be enabled, as explained in the documentation.

thanks for the reply.

yes, active_scan set to true (doing this via the gui).

also looking for output in the home-assistant_db file, not seeing what i can discern as temp/humidity information there either. kind of stumped :confused:

I have a couple of the Govee H5075 sensors. I am pretty sure these store historic data inside of them. Has anybody found a way to access this data (without the app of course)?

They do, up 20 days I think, but use a Bluetooth GATT service request, this component only monitors and decodes the broadcasted BLE data. There did not seem to be a need for this as Home Assistant does all of this without the complexity or battery drain of directly connecting to the device. I am not sure of any project that retrieves the historical data from the devices. Sorry.

I do not own a H5051 but do own a H5074 which should share the same encoding. I’ll see if I can reproduce the issue. What version of Home Assistant are you running and are you running any other Bluetooth components?

Are there any errors listed under Configuration->Logs?

no other bluetooth components running, running version 2021.8.5.

log after restart shows -

2021-08-12 09:52:07 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration hacs w
hich has not been tested by Home Assistant. This component might cause stability problems, be sure to
disable it if you experience issues with Home Assistant
2021-08-12 09:52:07 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration ble_mo
nitor which has not been tested by Home Assistant. This component might cause stability problems, be s
ure to disable it if you experience issues with Home Assistant
2021-08-12 09:52:12 ERROR (MainThread) [custom_components.hacs] avdeevsv91/ha_generic_hygrostat - Repo
sitory is archived.
2021-08-12 09:52:12 ERROR (MainThread) [custom_components.hacs] DavidFW1960/bom_forecast - Repository
is archived.
2021-08-12 09:52:12 ERROR (MainThread) [custom_components.hacs] bieniu/ha-gios - Repository is archive
d.

additional question - where are the configs for the device kept ? i’m looking in /config/custom_components/ble_monitor, and i cannot for the life of me figure out where the device specific stuff is at.

Do you mean where in the ble monitor settings? Open the options menu of ble monitor integration, select your MAC address in the devices dropdown menu and click submit. This should open the setting of your sensor.

Or do you mean where in HA. The config is stored in the registry, probably in config. It’s stored in a hidden folder (and should not be modified manually)

Try to enable debug logging, this should also reveal your config for ble monitor and possible issues. Instruction can be found in the faq.

The logs sometimes show errors that can prevent data from being updated while the rest of HA is working fine.

By “the configs”, there may be two possible answers. For hard coded configuration needed for some components, that is stored in /config/conguration.yaml (docs) but this component does not require hard coded configuration, except for maybe encryption keys. All of the data, layout, and settings are stored in a SQLite database or hidden files. but as @Ernst just said, it is best not to modify that.

At this point, I would try to to enable debug logging for ble_monitor by editing the configuration.yaml file and move the Govee device next to your HA setup to ensure that reception is not a problem.

thanks for the info on where the config is stored, was just trying to stare/compare at some other yaml configs i’d found online for pointers.

debug enabled, output thus -

2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor] Initializing BLE Monitor entry (config entry): <homeassistant.config_entries.ConfigEntry object at 0x7f8b9b5c0510>
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor] async_setup_entry: domain {}
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor] async_setup_entry: {‘bt_interface’: [‘5C:F3:70:A0:BA:29’], ‘period’: 60, ‘discovery’: False, ‘active_scan’: True, ‘decimals’: 1, ‘log_spikes’: False, ‘use_median’: False, ‘restore_state’: False, ‘report_unknown’: ‘Govee’, ‘devices’: [{‘mac’: ‘E3:60:59:21:E4:92’, ‘temperature_unit’: ‘°F’, ‘decimals’: 1, ‘use_median’: ‘default’, ‘restore_state’: True, ‘reset_timer’: 35, ‘track_device’: True, ‘tracker_scan_interval’: 20, ‘consider_home’: 180, ‘name’: ‘E3605921E492’}, {‘mac’: ‘EE:60:59:21:E4:92’, ‘temperature_unit’: ‘°F’, ‘decimals’: ‘default’, ‘use_median’: ‘default’, ‘restore_state’: ‘default’, ‘reset_timer’: 35, ‘track_device’: True, ‘tracker_scan_interval’: 20, ‘consider_home’: 180}], ‘is_flow’: True, ‘hci_interface’: [0]}
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor] HCI interface is [0]
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor] Spawning HCIdump thread
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor] HCIdump thread: Init
2021-08-12 13:14:25 INFO (MainThread) [custom_components.ble_monitor] Attention! Option report_unknown is enabled for Govee sensors, be ready for a huge output…
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor] 0 encryptors mac:key pairs loaded.
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor] sensor whitelist: [E3:60:59:21:E4:92, EE:60:59:21:E4:92]
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor] 2 sensor whitelist item(s) loaded.
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor] 2 device tracker(s) being monitored.
2021-08-12 13:14:25 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: Run
2021-08-12 13:14:25 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: connected to hci0
2021-08-12 13:14:25 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: start main event_loop
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Starting binary sensor entry startup
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] BLE binary sensors updater initialization
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] BLE binary sensors updater initialized
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Binary sensor entry setup finished
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Starting measuring sensor entry startup
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor.sensor] BLE sensors updater initialization
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor.sensor] BLE sensors updater initialized
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Measuring sensor entry setup finished
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Binary entities updater loop started!
2021-08-12 13:14:25 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Entities updater loop started!
2021-08-12 13:14:26 DEBUG (MainThread) [custom_components.ble_monitor.device_tracker] Starting device tracker entry startup
2021-08-12 13:14:26 DEBUG (MainThread) [custom_components.ble_monitor.device_tracker] BLE device tracker updater initialization
2021-08-12 13:14:26 DEBUG (MainThread) [custom_components.ble_monitor.device_tracker] BLE device tracker updater initialized
2021-08-12 13:14:26 DEBUG (MainThread) [custom_components.ble_monitor.device_tracker] Device Tracker entry setup finished
2021-08-12 13:14:26 DEBUG (MainThread) [custom_components.ble_monitor.device_tracker] Device tracker updater loop started!
2021-08-12 13:14:26 DEBUG (MainThread) [custom_components.ble_monitor.device_tracker] Device tracker device with mac address E3:60:59:21:E4:92 has the following settings. Name: E3605921E492. Restore state: True. Track device: True. Scan interval: 20. Consider home interval: 180.
2021-08-12 13:14:26 DEBUG (MainThread) [custom_components.ble_monitor.device_tracker] async_added_to_hass called for ble tracker E3605921E492
2021-08-12 13:14:27 DEBUG (MainThread) [custom_components.ble_monitor.device_tracker] Data device tracker received: {‘is connected’: True, ‘mac’: ‘E3605921E492’, ‘rssi’: -62}
2021-08-12 13:14:27 DEBUG (MainThread) [custom_components.ble_monitor.device_tracker] Data device tracker received: {‘is connected’: True, ‘mac’: ‘E3605921E492’, ‘rssi’: -59}
2021-08-12 13:14:28 DEBUG (MainThread) [custom_components.ble_monitor.device_tracker] Data device tracker received: {‘is connected’: True, ‘mac’: ‘E3605921E492’, ‘rssi’: -62}
2021-08-12 13:14:28 DEBUG (MainThread) [custom_components.ble_monitor.device_tracker] Data device tracker received: {‘is connected’: True, ‘mac’: ‘E3605921E492’, ‘rssi’: -59}

2021-08-12 13:15:26 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] 0 MiBeacon BLE ADV messages processed for 0 binary sensor device(s) total. Priority queue = 0
2021-08-12 13:15:26 DEBUG (MainThread) [custom_components.ble_monitor.sensor] 0 BLE ADV messages processed for 0 measuring device(s).
2021-08-12 13:15:26 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: main event_loop stopped, finishing
2021-08-12 13:15:26 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: Scanning will be restarted
2021-08-12 13:15:26 DEBUG (Thread-3) [custom_components.ble_monitor] 992 HCI events processed for previous period.
2021-08-12 13:15:26 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: Run
2021-08-12 13:15:26 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: connected to hci0
2021-08-12 13:15:26 DEBUG (Thread-3) [custom_components.ble_monitor] HCIdump thread: start main event_loop

Unless @Ernst knows of a better way to try this, can you edit the /config/custom_components/ble_monitor/ble_parser/govee.py line 39. It should currently be:

elif msg_length == 11 and device_id == 0xEC88:

Change it to

elif msg_length == 9 and device_id == 0xEC88:

save and restart Home Assistant?

Just starting with BLE devices, walking thru the instructions. Got a couple Govee H5101 temperature sensors.

The instructions say to install MiHome app, I did but can’t find my Govee devices. Tryed with US then changed to China, still not finding my devices. Should they be there?

It occurred to me that I should install Govee App instead and it shows both my sensors just not the MAC address.

Any advice on what I should do now?