MQTT disconnects - Tried keepalive but no go. ESP8266 Board

Hass running on an ubuntu install.
Mosquitto broken installed via plugin option.

I have 5 Sonoff’s that are running fine, but this newly setup ESP8266 board running my own custom code is seeing

1561960656: New client connected from 192.168.231.180 as ESP-Garage (p2, c1, k15, u'ESP-Garage').
1561960679: Client ESP-Garage has exceeded timeout, disconnecting.
1561960680: New connection from 192.168.231.180 on port 1883.
1561960680: New client connected from 192.168.231.180 as ESP-Garage (p2, c1, k15, u'ESP-Garage').
1561960703: Client ESP-Garage has exceeded timeout, disconnecting.
1561960704: New connection from 192.168.231.180 on port 1883.
1561960704: New client connected from 192.168.231.180 as ESP-Garage (p2, c1, k15, u'ESP-Garage').
1561960727: Client ESP-Garage has exceeded timeout, disconnecting.
1561960728: New connection from 192.168.231.180 on port 1883.
1561960728: New client connected from 192.168.231.180 as ESP-Garage (p2, c1, k15, u'ESP-Garage').
1561960751: Client ESP-Garage has exceeded timeout, disconnecting.
1561960752: New connection from 192.168.231.180 on port 1883.
1561960752: New client connected from 192.168.231.180 as ESP-Garage (p2, c1, k15, u'ESP-Garage').
1561960775: Client ESP-Garage has exceeded timeout, disconnecting.
1561960776: New connection from 192.168.231.180 on port 1883.
1561960776: New client connected from 192.168.231.180 as ESP-Garage (p2, c1, k15, u'ESP-Garage').

I’ve tried adjusting the timmer via
/////////////////////
// Inclusions //
/////////////////////
#define MQTT_KEEPALIVE 60
#include <PubSubClient.h>
#include <ESP8266WiFi.h>

But the console always shows that the ESP8266 board is reconnecting in loop()

Trying to adjust the logging level but seems i’m doing something wrong.

1561961290: New connection from 192.168.230.21 on port 1883.
1561961290: New client connected from 192.168.230.21 as mqtt-explorer-7ec56899 (p2, c1, k60, u'mqtt').
1561961290: New connection from 172.30.32.1 on port 1883.
1561961290: New client connected from 172.30.32.1 as auto-071F0043-FDA4-44DF-4D7A-2A9F98E9AE92 (p2, c1, k60, u'mqtt').
1561961291: New connection from 192.168.231.180 on port 1883.
1561961291: New client connected from 192.168.231.180 as ESP-Garage (p2, c1, k15, u'ESP-Garage').
1561961293: New connection from 192.168.231.203 on port 1883.
1561961293: New client connected from 192.168.231.203 as IoT-Sonoff-3 (p2, c1, k10, u'mqtt').
1561961293: New connection from 192.168.231.205 on port 1883.
1561961293: New client connected from 192.168.231.205 as IoT-Sonoff-5 (p2, c1, k10, u'mqtt').
1561961293: New connection from 192.168.231.201 on port 1883.
1561961293: New client connected from 192.168.231.201 as IoT-Sonoff-1 (p2, c1, k10, u'mqtt').
1561961294: New connection from 192.168.231.204 on port 1883.
1561961294: New client connected from 192.168.231.204 as IoT-Sonoff-4 (p2, c1, k10, u'mqtt').
1561961297: New connection from 192.168.231.10 on port 1883.
1561961297: New client connected from 192.168.231.10 as mqtt_c8a3c883.631a98 (p1, c1, k60, u'mqtt').
1561961297: New connection from 172.30.32.1 on port 1883.
1561961297: New client connected from 172.30.32.1 as Node-Red-MQTT (p1, c1, k60, u'mqtt').
1561961314: Client ESP-Garage has exceeded timeout, disconnecting.
1561961315: New connection from 192.168.231.180 on port 1883.
1561961315: New client connected from 192.168.231.180 as ESP-Garage (p2, c1, k15, u'ESP-Garage').
1561961338: Client ESP-Garage has exceeded timeout, disconnecting.
1561961339: New connection from 192.168.231.180 on port 1883.
1561961339: New client connected from 192.168.231.180 as ESP-Garage (p2, c1, k15, u'ESP-Garage').

Looks to me like an authentication error between the esp and mosquitto.

I tried incorrectly entering the password to see what the difference would be.

Mosquitto add says

1561961649: New connection from 192.168.231.180 on port 1883.
1561961649: Socket error on client <unknown>, disconnecting.

ESP Serial Output shows

Connected to ArrayNet
IP address: 192.168.x.x
Attempting MQTT connection...failed, rc=5 try again in 5 seconds
Attempting MQTT connection...failed, rc=5 try again in 5 seconds
Attempting MQTT connection...failed, rc=5 try again in 5 seconds
Attempting MQTT connection...failed, rc=5 try again in 5 seconds
Attempting MQTT connection...failed, rc=5 try again in 5 seconds
Attempting MQTT connection...failed, rc=5 try again in 5 seconds


OK, that is different, so not the problem.

When the password is connect, do you actually get any data appearing in mosquitto?

PS, are you aware of esphome? Pretty foolproof.

Yes, I get the initial on-alive message saying the system connected.

Also, every ~6 seconds the ESP board checks for motion and sends MQTT messages

image

Here is my loop()

void loop(){
    Serial.println("Entering loop()");
// First make sure MQTT is connected 
  {
  if (!client.connected()) {
    reconnect();           }
  }  //Close If

and the reconnect() function


void reconnect() {      // This is for the MQTT connection
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    if (client.connect(mqtt_sensorname, mqtt_user, mqtt_password)) {
      Serial.println("connected");
      client.subscribe(mqtttopic); 
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

I was getting errors like this on a self made firmware for the NodeMCU, I ended up going with Tasmota in the end, and it seems to be working OK now, it may be worth trying Tasmota as it works well with MQTT :slight_smile:

So it works, but disconnects every approx 23 seconds.

Yes, Everything is working, except the disconnect messages. Not sure the exact time, or where you saw 23 seconds.

As a quick update, I started sending MQTT messages when there was not Motion and it seems to be stable. I guess the receiving of a message resets the timeout…

So my issue is solved, but I don’t understand why the ESPP/MQTT keepalive/ping’s weren’t keeping the connection up.

ESPhome does look interesting, but I prefer my own custom flows

Hi,

It looks like you are using the ESP8266 pub-sub client library. If so, you can call the loop() function as described at https://pubsubclient.knolleary.net/api.html if you don’t want to disconnect. Here is the description of that function:

boolean loop ()
This should be called regularly to allow the client to process incoming messages and maintain its connection to the server.
Returns:

  • false - the client is no longer connected
  • true - the client is still connected
2 Likes

What @old_fogey said - I had the exact same issues until I figured out that you need to allow some time for the PubSubClient library to do its own processing (including sending keepalives to the server).

So make sure that you call the mqtt loop() regularly. And also include a delay() in your Arduino loop() function - this allows the ESP SDK to perform its own processing (including network processing).

Thanks @old_fogey and @bogd

I did notice that send messages in MQTT kept those messages down, but not fully. I was sending “off” messages to my MQTT server once ever 2 seconds but was still getting disconnects.

My quesiton now is, although I’m not using loop() I am using the following

void loop(){
    Serial.println("Entering loop()");
// First make sure MQTT is connected 
  {
  if (!client.connected()) {
    reconnect();           }
  }  //Close If

Wouldn’t that accomplish the same?

No, it does not accomplish the same.

Your (Arduino) loop() just loops as quickly as it can, unless your MQTT client is disconnected. So while the client is still connected, the loop() just… loops very quickly, prints to the serial, and loops again. As a result, the MQTT client does not get a chance to send its keepalives, and the ESP does not get a chance to do its housekeeping.

As mentioned before, you should add two things:
a) a client.loop() call for the MQTT client
b) a delay(500) (or 100, or whatever suits your application) for the ESP

[ Edited to add: also, what’s with the extra braces in your code? You have one set of {} that seem to serve no purpose… ]

Thanks @bogd

it did seem that just publishing a message in every loop() seemed to keep MQTT stable. Maybe just that few extra 100th of a second made the different? But I noticed it would go minutes between those connecting messages.

Now, with the client.loop() in there, it’s working perfect (alongside a delay)

Thanks a bunch!

I think that it was more stable when sending messages because there was actually some communication between your client and the MQTT broker, and that prevented the broker from declaring the connection lost.

Either way, glad to hear that it is working now!

Hi !

I am srtuggeling with the same issue. I am not sure, where to insert the client.loop.
Can you please help?

Here is my code:

#include <ESP8266WiFi.h>
#include <PubSubClient.h>

#define LISTEN_PORT 80

//Pins
int relay_pin1 = 12;
int temp_pin1 = A0;

// WIFI
const char* ssid = “SSID”;
const char* password = “WIFI password”;

//MQTT
const char* mqtt_server = “servers IP”;
const int mqttPort = 1883;
const char* mqttUser = “user”;
const char* mqttPassword = “password”;

WiFiClient espClient;
PubSubClient client(espClient);
WiFiServer server(LISTEN_PORT);

void callback(char* topic, byte* payload, unsigned int length) {
Serial.print(“Message arrived [”);
Serial.print(topic);
Serial.print("] ");
for (int i=0;i<length;i++) {
char receivedChar = (char)payload[i];
Serial.print(receivedChar);
}
Serial.println();
}

//RECONNECT
void reconnect() {
// Loop until we’re reconnected
while (!client.connected()) {
Serial.print(“Attempting MQTT connection…”);
// Attempt to connect
if (client.connect(“AKAC_ESP8266_1”, mqttUser, mqttPassword, “AKAC/ESP8266_1/Status”, 2, 1,“OFFLINE” )) {
Serial.println(“connected”);
// … and subscribe to topic
client.subscribe(“X”);
} else {

//RC descriptions

//-4 : MQTT_CONNECTION_TIMEOUT - the server didn’t respond within the keepalive time
//-3 : MQTT_CONNECTION_LOST - the network connection was broken
//-2 : MQTT_CONNECT_FAILED - the network connection failed
//-1 : MQTT_DISCONNECTED - the client is disconnected cleanly
//0 : MQTT_CONNECTED - the client is connected
//1 : MQTT_CONNECT_BAD_PROTOCOL - the server doesn’t support the requested version of MQTT
//2 : MQTT_CONNECT_BAD_CLIENT_ID - the server rejected the client identifier
//3 : MQTT_CONNECT_UNAVAILABLE - the server was unable to accept the connection
//4 : MQTT_CONNECT_BAD_CREDENTIALS - the username/password were rejected
//5 : MQTT_CONNECT_UNAUTHORIZED - the client was not authorized to connect

Serial.print(“failed, rc=”);
Serial.print(client.state());
if (client.state() == -4){Serial.print (“MQTT_CONNECTION_TIMEOUT - the server didn’t respond within the keepalive time”);}
if (client.state() == -3){Serial.print (“MQTT_CONNECTION_LOST - the network connection was broken”);}
if (client.state() == -2){Serial.print (“MQTT_CONNECT_FAILED - the network connection failed”);}
if (client.state() == -1){Serial.print (“MQTT_DISCONNECTED - the client is disconnected cleanly”);}
if (client.state() == 0){Serial.print (“MQTT_CONNECTED - the client is connected”);}
if (client.state() == 1){Serial.print (“MQTT_CONNECT_BAD_PROTOCOL - the server doesn’t support the requested version of MQTT”);}
if (client.state() == 2){Serial.print (“MQTT_CONNECT_BAD_CLIENT_ID - the server rejected the client identifier”);}
if (client.state() == 3){Serial.print (“MQTT_CONNECT_UNAVAILABLE - the server was unable to accept the connection”);}
if (client.state() == 4){Serial.print (“MQTT_CONNECT_BAD_CREDENTIALS - the username/password were rejected”);}
if (client.state() == 4){Serial.print (“MQTT_CONNECT_UNAUTHORIZED - the client was not authorized to connect”);}

Serial.println(" try again in 5 seconds");
// Wait 5 seconds before retrying
delay(5000);
}
}
}

//SETUP
void setup() {
// put your setup code here, to run once:
Serial.begin(115200);
client.setServer(mqtt_server, mqttPort);
client.setCallback(callback);

//PINMODES
pinMode(relay_pin1, OUTPUT);
pinMode(temp_pin1, INPUT);

}

//LOOP
void loop() {
// put your main code here, to run repeatedly:
if (!client.connected()) {
reconnect();
}
client.loop();
client.publish(“X”, “online”);
delay(1000);
}

Your client.loop() seems to be in the right place (so that shouldn’t be the problem). The other thing to check is that your mqtt.client name is unique on your network as it will disconnect each time it loops if a different device has the same name.

In your sample code AKAC_ESP8266_1 must be unique.

You mean, there should be not other device connected, or trying to connect with the same client ID?

That is not the case, there is only one client named “AKAC_ESP8266_1” in my system. Every other client has his own unique name, client ID.

However they all login with the same user/pass to the broker, but that should be no problem I guess.

First of all most of the problems with mqtt disconnecting is because bugged esp core. Simply not working good with some wifi chips. The most stable are Core 2.3 and the newest, prerelease 2.6.x. Forget about core 2.4 and 2.5…