MQTT Saving in-memory database to /data/mosquitto.db

Configuration
1

I have HASSIO setup on my Raspberry pi and everything works great, I can control things, turn my lights on and off, however there is one issue I have been dealing with for a while and I cannot seem to figure it out.

I am using Bruh automation for the WS2811 LED setup. I have it working, but it will only stay on for a random amount of time then recieves the following message: bruh/mqttstrip/setpowerpub: b’OFF’

As you can see in the logs below, it was at 23:13:33, then 01:02:57 and finally it lasted all the way until 06:46:26 before it received another turn off message.

I have included my setup below, and some of the logs from MQTT I also changed the IP addresses and usernames.

If anyone is able to assist me with this issue, I would much appreciate it. Let me know if I need to provide any other info.

Hassio MQTT:

{
“plain”: true,
“ssl”: false,
“anonymous”: false,
“logins”: [
{
“username”: “my-username”,
“password”: “my-password”
}
],
“customize”: {
“active”: false,
“folder”: “mosquitto”
},
“certfile”: “fullchain.pem”,
“keyfile”: “privkey.pem”
}

Configuration.yaml
mqtt:
broker: MyIpaddress
port: 1883
username: my-username
password: my-password
discovery: true

Hassio Log:

[homeassistant.components.mqtt] Transmitting message on bruh/mqttstrip/setpower: OFF
2018-11-18 23:13:33 DEBUG (MainThread) [homeassistant.components.mqtt] Received message on bruh/mqttstrip/setpowerpub: b’OFF’
2018-11-19 01:02:57 DEBUG (MainThread) [homeassistant.components.mqtt] Received message on bruh/mqttstrip/setpowerpub: b’OFF’
2018-11-19 06:46:26 DEBUG (MainThread) [homeassistant.components.mqtt] Received message on bruh/mqttstrip/setpowerpub: b’OFF’

MQTT log:

[INFO] Setup mosquitto configuration
[INFO] Found local users inside config
[INFO] Initialize Hass.io Add-on services
[INFO] Initialize Home Assistant discovery
[INFO] Start Mosquitto daemon
1542599598: mosquitto version 1.4.15 (build date 2018-03-04 15:35:59+0000) starting
1542599598: Config loaded from /etc/mosquitto.conf.
1542599598: *** auth-plug: startup
1542599598: ** Configured order: http
1542599598: Opening ipv4 listen socket on port 1883.
1542599598: Opening ipv6 listen socket on port 1883.
1542599598: Opening websockets listen socket on port 1884.
1542599599: Opening ipv4 listen socket on port 8883.
1542599599: Opening ipv6 listen socket on port 8883.
1542599599: Opening websockets listen socket on port 8884.
1542599599: Warning: Mosquitto should not be run as root/administrator.
1542599603: New connection from WEMOS-D1Mini_IP on port 1883.
[INFO] found my-username on local database
1542599603: New client connected from WEMOS-D1Mini_IP as Lights (c1, k60, u’my-username’).
1542599890: New connection from MQTTBrokerIP on port 1883.
[INFO] found my-username on local database
1542599890: New client connected from MQTTBrokerIP as MQTT (c1, k60, u’my-username’).
1542601399: Saving in-memory database to /data/mosquitto.db.
1542603200: Saving in-memory database to /data/mosquitto.db.
1542605001: Saving in-memory database to /data/mosquitto.db.
1542606802: Saving in-memory database to /data/mosquitto.db.
1542608603: Saving in-memory database to /data/mosquitto.db.
1542610404: Saving in-memory database to /data/mosquitto.db.
1542610977: New connection from WEMOS-D1Mini_IP on port 1883.
[INFO] found my-username on local database
1542610977: Client Lights already connected, closing old connection.
1542610977: Client Lights disconnected.
1542610977: New client connected from WEMOS-D1Mini_IP as Lights (c1, k60, u’my-username’).
1542612205: Saving in-memory database to /data/mosquitto.db.
1542614006: Saving in-memory database to /data/mosquitto.db.
1542615807: Saving in-memory database to /data/mosquitto.db.
1542617608: Saving in-memory database to /data/mosquitto.db.
1542619409: Saving in-memory database to /data/mosquitto.db.
1542621210: Saving in-memory database to /data/mosquitto.db.
1542623011: Saving in-memory database to /data/mosquitto.db.
1542624812: Saving in-memory database to /data/mosquitto.db.
1542626613: Saving in-memory database to /data/mosquitto.db.
1542628414: Saving in-memory database to /data/mosquitto.db.
1542630215: Saving in-memory database to /data/mosquitto.db.
1542631585: Socket error on client Lights, disconnecting.
1542631586: New connection from WEMOS-D1Mini_IP on port 1883.
[INFO] found my-username on local database
1542631586: New client connected from WEMOS-D1Mini_IP as Lights (c1, k60, u’my-username’).
1542632016: Saving in-memory database to /data/mosquitto.db.
1542633817: Saving in-memory database to /data/mosquitto.db.
1542635618: Saving in-memory database to /data/mosquitto.db.
1542637419: Saving in-memory database to /data/mosquitto.db.
1542639220: Saving in-memory database to /data/mosquitto.db.
1542641021: Saving in-memory database to /data/mosquitto.db.
1542642822: Saving in-memory database to /data/mosquitto.db.
1542644623: Saving in-memory database to /data/mosquitto.db.

2

Do the messages occur at the same time as the disconnections in the mosquitto log, or are they completely unrelated?

3

The new client connected seem to correlate with the Off message, but I cannot find a way to tell the exact times of each. In MQTT it just gives that number, the hassio log shows a date and time. is there another log for MQTT besides inside hassio?

4

You can convert the time in the mosquitto log using the date command

$ date -d @1542631586
Mon 19 Nov 12:46:26 GMT 2018

I can’t see any correlation to the times in the HA log in your first post, but that may be a timezone effect.

I suggest, to start with, that you remove the power from the light, check to see if the message is sent - it could be the broker sending a will message, and then reconnect the light, and check again to see if the message is sent at that point.

5

It keeps reconnecting to the MQTT broker.

What if the D1 Mini is restarting itself? Upon restart, it would attempt to reconnect to the broker. Upon restart, what is the initial state for its relay? Perhaps off?

If it’s not restarting itself but simply losing its WiFi connection to the broker, is there a possibility it initializes its relay to off?

You can answer some these questions by performing a few experiments like power-cycling the D1 while its attached lights are on and either blocking WiFi reception (with something metallic) or simply shutting off WiFI and then turning it back on. Basically, you’re trying to replicate a scenario that causes the D1 Mini to reconnect to the broker and observe if it turns off its load in the process.

6

Thanks for the feedback. I am not sure if the d1 mini is restarting itself, is there a way to see live traffic?

Yes, the initial state is off. So if something goes wrong in the middle of the night it’s not on all night.

It reconnects back to the broker, but when it’s on, why does it go through this disconnect? or restart itself, and how can i prevent it from restarting itself?

Does anyone know an easy way to make it resume it’s previous state if it reboots or power failure or what not. that could probably help out.

when I reboot the D1 mini and there is no wifi, the lights go on solid red, and never change until it reconnects back to the wifi. Then the lights go back into an off state.

2018-11-19 16:43:12 INFO (Thread-2) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
2018-11-19 16:43:12 DEBUG (MainThread) [homeassistant.components.mqtt] Subscribing to bruh/mqttstrip/setbrightnesspub
2018-11-19 16:43:12 DEBUG (MainThread) [homeassistant.components.mqtt] Subscribing to bruh/mqttstrip/setcolorpub
2018-11-19 16:43:12 DEBUG (MainThread) [homeassistant.components.mqtt] Subscribing to bruh/mqttstrip/setpowerpub
2018-11-19 16:43:12 DEBUG (MainThread) [homeassistant.components.mqtt] Subscribing to homeassistant/#

7

Thank you for showing me that. I am going to try and correlate the times with my logs.

And it stopped again, right when the D1 mini connected again:

1542681300: Saving in-memory database to /data/mosquitto.db.
1542681399: New connection from MQTTIp address on port 1883.
[INFO] found jsnarey on local database
1542681399: Client Lights already connected, closing old connection.
1542681399: Client Lights disconnected.
1542681399: New client connected from 192.168.1.7 as Lights (c1, k60, u’jMyusername’).
1542683101: Saving in-memory database to /data/mosquitto.db.

core-ssh:~# date -d @1542681300
Mon Nov 19 20:35:00 CST 2018

core-ssh:~# date -d @1542681399
Mon Nov 19 20:36:39 CST 2018

2018-11-19 20:36:39 DEBUG (MainThread) [homeassistant.components.mqtt] Received message on bruh/mqttstrip/setpowerpub: b’OFF’

8

That’s the root question.

Does your WiFi access point or router maintain a system log? That’s where I’d check to see if there’s a record of the device disconnecting/reconnecting from the WiFi network. If it is, then is it in a location with a weak signal? There are phone apps that can measure and report WiFI signal strength.

9

Wifi connections are always subject to interference, so your system has to be able to handle disconnections correctly.

And you also have to find out why it is disconnecting so frequently.

To find what is happening when it disconnects, you have to do some manual testing to see when the message is generated, and then look at your sketch to see what is generating the message.

I think testing the signal strength is a good idea, but there may also be problems with your firmware causing a crash, or possibly faulty hardware. The only way I can see to check the latter two would be to connect it up to a monitor to see the console output.

10

How would you recommend setting up a monitor to see console output, I checked my wifi, and it looks like the device connected again at about the same time it turned off. Nov 19,2018 18:36:36 So it could very well be interference with my wifi, and it looses connection. It’s also hard to tell how good the wifi is, I used my Galaxy Note 9 with wifi analyzer and it shows I have good wifi signal where my d1mini is located, however my radio on the galaxy note 9 is way better than a d1mini. I also have a NodeMCU that I have tested and it seemed like it had more frequent disconnections. I have a couple more NodeMCU Lua chips on order I am going to test. Maybe I just need a stronger router to boost the signal to the chips. Is there a router that anyone can recommend that wont break the bank?

11

So one thing I just thought of, I am powering this thing from the VCC 5v pin, should I be powering this using 3.3 volt? Could that be the cause of it reconnecting to wifi… If so I can get a buck converter to convert my 5V down to 3.3…

12

Sorry, I don’t know what you mean by ‘console output’. If you mean to directly monitor the NodeMCU’s operation, I don’t know how to do that. My guess is probably through the NodeMCU’s serial interface but I have no practical experience with that.

When you say “good” what does that mean in dBm units?

Perhaps, but we’re looking for an obvious lack of signal strength. If we’re not finding it then the outages may be due to something else, namely something random and harder to isolate.

Obviously not a comforting discovery. It suggests there’s an inherent stability problem when operating in your WiFi environment.

Maybe but disconnections can still occur even in a good WiFi operating environment.

If I was experiencing this issue, here’s how I would handle it. After determining that the WiFi signal is ‘good enough’ for most other devices, except this one, I would try another one (with the same firmware) in the same spot. If it misbehaved the same way then I’d conclude it’s not specific to one NodeMCU but all (with the same firmware).

At this point, I’d resign myself to the fact that, even in an excellent WiFi environment, there’ll be the occasional disconnection. For example, an intentional restart of the WiFi access point will cause all WiFi clients to disconnect and reconnect. So I’d focus on making the device correctly handle its own state upon reconnection.

If the device, upon reconnection, currently sets its load to off by default, that’s something I’d want to improve. Ideally, it should set it to the ‘last-known state’. If it was off, it should remain off; if it was on, then it should remain on. In this manner, the load’s state is immune to momentary interruptions in WiFi connection. In fact, if the ‘last-known state’ can be stored in way that it survives a reboot (i.e. in non-volatile memory) then the device can re-establish the correct state even after power-cycling.

I believe much of what I have described is already possible with open-source firmware like Tasmota, Espurna, etc. If you are using custom firmware then you’ll need to modify it yourself to achieve my so-called ‘ideal’ behavior.

Anyway, that’s how I’d tackle the problem; concede that WiFi disconnections can happen and make the system resilient to them (i.e. fault-tolerant).

13

Thanks for everyone’s feedback, I think I figured it out. I added retain: true to my Matt topic and the last time it stopped it started right back up after it reconnected. So I’m going with them for now.

14

Where did you add it?

  1. If you added it the device’s side of the equation, that’s OK.
  2. If you added it to Home Assistant, it’s not.

If the answer is 2, there are many posts on this forum explaining why it’s inadvisable to set retain: true for entities that publish a command_topic. It’s a common practice to fix random disconnects but it has a hidden flaw that often reveals itself at the worst possible times (like garage doors opening themselves in the middle of the night right after a power-failure … and chances are you’ll be away when it happens).

1 Like
15

Yeah, for a normal light or garage door I would not want it set, but for this setup, it’s okay. I only have my holiday lights hooked up to it, and I have another automation set to shut off the 8266, and its automated so it should be okay for now. I never had the random disconnects with my sonoff which is odd, but oh well. Thanks again for the responses.

1 Like
16

I’ve been tracking down a similar issue.

Try adding WiFi.setSleepMode(WIFI_NONE_SLEEP); to you void setup() for your WiFi. If I had to guess, I’d imagine it isn’t there now – but I’ve been wrong before.

Going on 10 hours, and it’s cured my woes so far – hopefully it will work for you, too.

17

Thanks for the reply, I had it already. What I have now seems to be working for me.

I recently moved my stuff over to proxmox VM and have google wifi, I have not seen as many disconnections.

void setup() {
WiFi.setSleepMode(WIFI_NONE_SLEEP);
Serial.begin(115200);

1 Like