MQTT/hot tub/something happening the same time each night

Hi all,

I am using HA 2022.7.2 with Mosquitto 6.1.2 inside a proxmox VM. I have a hot tub connected via this library running on another VM, which talks to the Mosquitto running on the HA VM. Everything is working well. Kinda.

At 8:49:xx every night, all the hot tub controls become “unavailable”. The only way I can get it back working is by restarting the aforementioned library service on the other VM.

Some clues:

  1. There are no automations running at that time. In the logbook I can only see messages " became unavailable".
  2. xx (the number of seconds when the problem occurs) varies throughout the minute, making me think it’s not automation?
  3. The hot tub topics on the MQTT server still change when I manually operate the tub controls, making me think it’s not the hot tub library? (checked with MQTT Explorer)
  4. All the other stuff happening on the MQTT server carries on without a problem.
  5. The systemctl logs for the hot tub library don’t indicate anything unusual at that time.

Is there any other kind of digging around I can do to figure this out?

Thanks!

Have you monitored the topic using MQTTExplorer to see what’s happening at that time?
What other devices do you have on MQTT?

Yeah I have, sorry should have made that a bit clearer in point #3, I’ll edit it. I was using MQTT Explorer to monitor the MQTT server. I couldn’t see anything unusual happen at that time. The hot tub still updates topics in MQTT, there’s something that causes them to suddenly appear unavailable in HA. I checked out the history on a bunch of these topics as well, can’t see anything unusual happening either.

I have a bunch of DIY stuff talking to the same MQTT server that has no problem. Some dimmers, current sensors and a couple of other bits and pieces. It’s all in different topics though. The library sets up its own path in “homie/bwa” which has a lot of sub-paths for different parts of the hot tub.

What hardware are you running the Balboa app on?
Is there any chance you are running it on another processor at the same time?
Is the Balboa sending a status update periodically?

Hi Steve,

Balboa app hardware is a proxmox VM. 1CPU, 1G RAM, Ubuntu 20.04. USB → RS485 converter connected to the tub control board. The VM host is i5-3470, 16GB RAM.

I only ever ran the balboa app on this VM and my laptop. My laptop caught fire last week(!) so that’s out of the picture. I did check ps aux | grep ruby on the VM and only one process is running. I don’t think it’s possible to run two anyway as the first one will grab the serial port.

The app only sends status updates via MQTT when parameters change on the tub. That does seem to still be working when the controls in HA are unavailable, as item #3 in the original post. I haven’t tried the reverse yet - i.e. setting one of its parameters manually in MQTT Explorer and see if the tub responds.

What I have figured out is that the balboa app uses the “homie/spa/” topic for all the “live” parameters, and then adds MQTT Discovery data to all the component types under “homeassistant/”. Maybe that’s getting broken somehow. Still learning about that!

I will also try spinning up another HA VM where I can mess around with the clock and see what the significance of 8:49PM is without messing up all the stuff runing on my main HA.

One thing I’ve observed that can make an MQTT device “unavailable” is if HA doesn’t get a status update from the device. At 8:49, how long has it been since the last status message? At 8:48 can you send a dummy status message from MQTT explorer?

The tub is pretty active in the couple of hours prior to 8:49. I have it starting a filter cycle and re-heating at 7PM, so the pump + heater is active and the temperature is being updated as it climbs.

At 7AM every morning, when power gets expensive, I store the current target temperature in an input_number and set the tub temperature low, like 30°, so it never heats until 7PM when power becomes cheap again and I restore the stored target temperature.

During this daytime I only get updates every two hours, when the pump kicks on for one minute and the temperature gets updated. But all the controls remain active throughout.

I’ve tried adjusting the time-related controls on the tub, like the time/lengths of the filter cycles, thinking that it might be screwing up “some time after x happens” but nope! The witching hour still persists at 8:49.

Tonight I’ll try jabbing some buttons at 8:48 and causing some activity just in case.

Ultimately it doesn’t hurt anything when I restart the systemd service, so I could just do it with a cron job, or even have an automation that monitors the hot tub device and restarts the service if it goes unvailable. Still, it would be nice to solve it “properly” though!

HI Steve, I think it’s the Balboa library after all. It happened at 20:49:41 this time! I did mess with the controls (both on the tub and HA) at 20:30 to make sure everything was still working.

Looking at the MQTT logs, the topic homie/bwa/$state which is normally set to “ready” gets set to “init” then “ready” then “lost” within the space of a couple seconds. All the hot tub-related topics under homeassistant/sensor, homeassistant/switch etc are looking at homie/bwa/$state to equal “ready” as their availability_topic. Once it switches to “lost” that’s when the HA controls go unavailable.

As I found before, the physical tub controls still change topics in the MQTT server, so the balboa app is still somewhat working. If I manually change the homie/bwa/$state topic back to “ready”, all the controls come back in HA, and work.

Since I can’t see anything weird in the balboa app logs, I’ll leave a note on their github and see if I can get some advice there.

Many thanks for your input though. It’s prompted me to learn about MQTT Discovery properly today anyway!

As a workaround until Balboa fixes their problem, you could make an automation to set the topic to “ready” at 20:50 every night. Personally, I would do it in Node Red with three nodes. (Timer, Change, Mqtt OUT). I haven’t ever tried sending MQTT messages from Home Assistant.

Hi Steve, I’m sure it turned out to be something on my end. I’m still not sure what though - It fixed itself.

I posted the github issue here. It’s not even balboa’s thing, this was made by a very kind gentleman who has gone to all the trouble of figuring out the serial protocol and now I can operate the tub with a $5 USB serial adapter. The official solution is a $700 wi-fi interface that only talks to their smartphone app…

I have left my workaround in place, which is an automation that monitors the status of one of the tub’s HA entities. When it goes “unavailable” it logs into the balboa VM via SSH and restarts the service. It hasn’t run in days though. Fingers crossed! I might start from scratch with a completely fresh linux install for the balboa VM, but for now it’s good!

Thanks for your help Steve!

They only need to sell one to recoup their engineering costs.
I am from California where all you need is a half-wine cask, a heater and a pump.