How best to deal with MQTT sensor name changing after battery change

Configuration
1

I have a number of “cheap” 433MHz temperature & humidity sensors that I track using a LILYGO® LoRa32 V2.1 ESP32 SX1278 (433MHz) board to detect and decode the 433MHz signals and then use MQTT to auto-discover and publish the corresponding topics.

However, when I change the battery in the sensor, the sensor seems to pick a random (new) ID which results in a new MQTT topic and more importantly, a new sensor name.

I have somewhat and painfully managed to “fix” this manually by manipulating the SQLite DB and modifying various files in /homeassistant/.storage that reference the sensor.

However, this is all painful, kludgy and likely brittle as it would be easy to make a mistake…

Is there a better (and ideally automated) way to handle battery changes that would allow me to continue to use the old sensor name without losing any of the history data. or requiring changes to various scripts and automations that reference the old sensor name.

2

Change the entity_id back to what it was before.

3

But I want to have the same ‘metadata_id’ since I have othe SQLITE scripts outside of HomeAssistant that for example reference the ‘statistics’ or ‘states’ table. And I don’t want to constantly have to join the ‘statistics_meta’ and ‘states_meta’ tables to select sensors.

So I would like the sensor to have the same ‘metadata_id’ rather than adding more and more sensors each time the battery is chamged.

4

I found a simple and relatively elegant solution:

  1. First, I used my rename_device+entities script (see Why do I get error setting state_class 'measurement' and device_class 'volume' - #11 by puterboy) to give the relevant devices meaningful names and remove the channel-specific portion of the name.
    This routine allows one to change all the name specific components that include the channel (including friendly_name, name_by_user, unique_id, etc.). Similarly for the MQTT discovery topic and message. This way there is no channel dependency in the internal naming conventions.
    For example, I renamed LaCrosse-TX141THBv2-1-53 to LaCrosse-TX141THBv2-KitchenFridge
    This only needs to be done once to create channel-invariant naming – though it is not strictly necessary, if you are OK with having legacy channel names present in your device and entity names even as the actual numbers change when you change your battery.

  2. In the MQTT discovery message, I set the state topic to coincide with the existing channel number.
    "stat_t": "+/+/RTL_433toMQTT/LaCrosse-TX141THBv2/1/153"
    This can be done automatically as part of the above rename_device+entities command or can be done manually using MQTT Explorer or mosquitto_pub or the HA service mqtt.publish

  3. Then any time the channel number changes (e.g., after changing batteries), all you need to do is repeat #2.

Note that if you are using something like rtl_433_esp32 you need to shut off auto-discovery so that you don’t get duplicate new sensors created corresponding to the default entity name containing channel numbers.

5

Actually, after a battery change, all I need to do is run the following bash 1-liner:

(ORIG="3/200"; NEW=3/126; PREFIX=homeassistant; HOST=homeassistant; USER=MQTT; PWD='<PASSWORD>'; IFS=$'\n'; for line in $(timeout 1 mosquitto_sub -v -h "$HOST" -u "$USER" -P "$PWD" -t "$PREFIX/#" | sed -n  "s#\(\"\(stat_t\|state_topic\)\":\".*\)\($ORIG\)\"#\1$NEW\"#p"); do topic=${line%%/config *}; message=${line#*/config }; mosquitto_pub -h "$HOST" -u "$USER" -P "$PWD" -t "${topic}/config" -m "$message" -r; done)

The one-liner changes the discovery topics for all the entities corresponding to the device with original channel 3/200 to point to the same device and entities but with the new channel 3/126.

Format assumes you are using OpenMQTT Gateway.

Change the password to the password applicable to you.
Also, change the HOST hostname if needed.

1 Like
6

We changed the batteries in one of our sensor so are trying to figure out what we have to do to run your bash script.

First question where do you find the value for ORIG and NEW?

We are running MQTT Broker add on and MQTT integration. Is this considered “OpenMQTT Gateway”?

Would this bash 1-liner be run from a terminal window?

We did search for a file called mosquitto_pub and _sub (under the assumption that you are modifying these files) but we didn’t find it.

Any help would be greatly appreciated.

7

Note that the data is in the format of:
ID/Channel

  • The ID is typically 0-511 or 0-127 – it changes when the battery is changed
  • The Channel though shouldn’t change and is typically a 2-bit number (the above applies to the devices I use)

You can find the values of OR IG in multiple places.
Using the CLI:

  • /homeassistant/.storage/core.device_registry
  • Also in core.entity_registry, core.restore_state or anywhere device or entity name is mentioned
  • ``states_metatable ofhome-assistant_v2.db`

Using the GUI GUI, you can look under devices or entities Under Settings->Devices & Services

To find the value for “NEW”
Assuming you have MQTT discovery shut off so that you haven’t created new devices and entities for the sensors, you can use MQTT Explorer either as an add-on to HA or as a standalone app to see the name of the device being posted to your MQTT broker. Alternatively, you can run ` mosquitto_sub -h “$HOST” -u “$USER” -P “$PASSWD” -t ‘home/#’ to see all the devices being posted. You may need to wait a few minutes for your sensor to post data.

mosquitto_sub and mosquitto_pub should be part of the Mosquitto Broker add-on. I access it using the Advanced SSH add-on.

If you are not familiar and comfortable with BASH and SSH, I would advice caution before running such a 1-liner.

Also I would suggest using MQTT Explorer to make sure the format of your device names matches (and especially the device id & channel) match the format assumed in my 1-liner.

NOTE: If you have changed the battery and accidentally left discovery on, then assuming you haven’t collected too much data yet under the new device, you may want to delete the new device using my delete_device routine (see 6 Routines to Delete/Rename/Move Devices & Entities and their corresponding registry entries, data, and metadata - #7 by puterboy). Just be sure to shut off auto-discovery first. After deleting the newly created devices, use the rename_device+entities routine to rename the old device plus associated entities.

If you have already collected a lot of data under the new device, then you can manually use sql CLI commands to move the data from the old metadata_id to the new one (you would need to do this for the states, statistics, and statistics_short_term tables) – this is all manual SQL fu so be VERY careful if you have to go this path.

If you are using my scripts make sure to look at the VARIABLES section of each script to see if you need to change any variables. Also, make sure you have the latest versions.
Better yet, scan the script to make sure you understand and are comfortable with all it does.

BEWARE WHILE THE SCRIPT HAS MANY CHECKS AND BY DEFAULT CREATES COPIOUS BACKUPS, IT MODIFIES THE HA DB AND REGISTRY FILES SO IT COULD CAUSE IRREPARABLE DAMAGE TO YOUR SYSTEM, REQUIRING RESTORE FROM BACKUP.

1 Like
8

Thanks so much @puterboy for the great explanation. That really clears up our confusion.