MQTT no callback - PubSubClient - ESP32

Good evening Homies,

Trying to implement MQTT with a ESP32.
running:

Mosquitto broker
Home assistant
Host system
Hostname	raspberrypi
System	Raspbian GNU/Linux 10 (buster)

Running client.publish("xyz", buffer, n); it sends the data across perfectly and I can grab that
but when I try to use a callback on the other hand void callback(char* topic, byte* payload, unsigned int length) I’m getting nothing.

I’ve tried to work it out using the MQTT Settings listen to a topic in the configuration menu and can see it being pressed:

Message 12 received on /office/light1/switch/ at 21:36:
ON
QoS: 0 - Retain: false
Message 11 received on /office/light1/switch/ at 21:36:
ON
QoS: 0 - Retain: false

My yaml is:
light:

  • platform: mqtt
    name: “Office light”
    state_topic: “office/light1/status”
    command_topic: “office/light1/switch”
    payload_on: “ON”
    payload_off: “OFF”
    optimistic: false
    switch:
  • platform: mqtt
    name: “Example_Switch”
    state_topic: “room/light”
    command_topic: “room/light”
    payload_on: “on”
    payload_off: “off”

At this point, a little lost for what to try next. Any ideas or pointers would be great, i see from the forum this issue comes up a lot. I’ve tried including the delay(100); after the 1client.loop()1 as suggested in a few of the other post but no luck.

feel like it’s going to be a something small as the logs show up fine with the connection and i can receive it with no error.

Cheers.

Copy of code:

#include <ArduinoJson.h>
#include <WiFi.h>
#include <PubSubClient.h>
#include <WiFiManager.h>

WiFiClient espClient;
PubSubClient client(espClient);

unsigned long updateCurrentMillis;
unsigned long lastUpdateDelay;
unsigned long updateDelay = 1000;

// MQTT Network
const char* mqtt_server = "192.168.0.200";
//Node ID
const char *ID = "brainESP32";  // Name of our device, must be unique
//Topics
const char *TOPIC = "room/light";  // Topic to subcribe to
const char *STATE_TOPIC = "room/light/state";  // Topic to publish the light state to

// Home Assistant Credentials
const char *HA_USER = "********";
const char *HA_PASS = "*******";


void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
  setup_wifi(); // Connect to network
  client.setServer(mqtt_server, 1883);
  client.setCallback(callback);
}

void callback(char* topic, byte* payload, unsigned int length) {
  String topicStr = topic;
  Serial.print("Topic: ");
  Serial.println(topicStr);
  if (topicStr == "/office/light1/switch/") {
    //turn the switch on if the payload is '1' and publish to the MQTT server a confirmation message
    if (payload[0] == 'ON') {
      Serial.println("light on");
      client.publish("/office/light1/status/", "ON");
    }

    //turn the switch off if the payload is '0' and publish to the MQTT server a confirmation message
    else if (payload[0] == 'OFF') {
      Serial.println("light off");
      client.publish("/office/light1/status/", "OFF");
    }
  }
}

void loop() {
  if (!client.connected()) {
    reconnect();
  }
  client.loop();
  delay(100);
  updateCurrentMillis = millis();
  if (updateCurrentMillis - lastUpdateDelay >= updateDelay) {
    lastUpdateDelay = updateCurrentMillis;
    switchOut();
  }
}

// Reconnect to client
void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect(ID, HA_USER, HA_PASS)) {
      Serial.println("connected");
      Serial.print("Publishing to: ");
      Serial.println(ID);
      Serial.println('\n');
    }
    else {
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

void setup_wifi() {
  Serial.print("\nConnecting to ");
  //WiFiManager
  WiFiManager wifiManager;
  Serial.println("started");
  //wifiManager.resetSettings(); // turn off after intital settingfs are done
  wifiManager.setCustomHeadElement("<style>html{filter: invert(40%); -webkit-filter: invert(40%);}</style>");
  WiFiManagerParameter custom_text("<p>This is just a text paragraph</p>");
  wifiManager.addParameter(&custom_text);
  wifiManager.setAPStaticIPConfig(IPAddress(100, 100, 100, 100), IPAddress(100, 100, 100, 100), IPAddress(255, 255, 255, 0));
  wifiManager.autoConnect();
  
  while (WiFi.status() != WL_CONNECTED) { // Wait for connection
    delay(500);
    Serial.print(".");
  }
  Serial.println();
  Serial.println("WiFi connected");
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());
}

void switchOut() {
  const size_t capacity = JSON_OBJECT_SIZE(26);
  DynamicJsonDocument doc1(capacity);
  DynamicJsonDocument doc2(capacity);
  doc1["pbrAM"] = 0;
  doc1["pbrSS"] = 0;
  doc1["lightAM"] = 0;
  doc1["lp_1"] = 0;
  doc1["lp1_1"] = 0;
  doc1["lp1_2"] = 0;
  doc1["lp1_3"] = 1;
  doc1["lp1_4"] = 0;
  doc1["lp_2"] = 0;
  doc1["lp2_1"] = 1;
  doc1["lp2_2"] = 0;
  doc1["lp2_3"] = 0;
  doc1["lp2_4"] = 0;
  char buffer[256];
  size_t n = serializeJson(doc1, buffer);
  client.publish("brainOut1", buffer, n);
  doc2["tempAM"] = 0;
  doc2["tempSS"] = 1;
  doc2["airAM"] = 0;
  doc2["airSS"] = 0;
  doc2["doseAM"] = 0;
  doc2["doseSS"] = 1;
  doc2["dosephU"] = 0;
  doc2["dosepHD"] = 0;
  doc2["doseNut"] = 0;
  doc2["doseSam"] = 0;
  doc2["doseFill"] = 0;
  doc2["harvsetAM"] = 0;
  doc2["harvestSS"] = 0;
  n = serializeJson(doc2, buffer);
  client.publish("brainOut2", buffer, n);
}

After you connect to the MQTT server you need to subscribe to the topics of interest

So in your code after successful connect add this

client.subscribe("/office/light1/switch");

That should do it

1 Like

Thank you for that @KennethLavrsen, will give it a crack first thing tomorrow morning :),

Is there a best practice for placement? i.e. as you said after it has made a sucseful connection so therefor:

if (client.connect(ID, HA_USER, HA_PASS)) {
      boolean test = client.subscribe("/office/light1/switch/");
      Serial.println(test); //firgured i could check if it returns true or false to help further debug.

Either way will play around and update tomorrow. Thanks again

Additionally, do not publish within a callback. PubSubClient documentation warns against this since the topic and payload variables are used in the library for both functions. Publishing in a callback can lock up the library or have other unintended results.

Set a flag in your callback and then process that flag in the loop to perform your publish.

1 Like

@KennethLavrsen @AaronCake Thank you both for that, I believe it was a combination of a few things, and as it comes up on the forum 50 times without a solution I’m going to try and explain it so when I forget 3 weeks later I can do it again… :sweat_smile: :sweat_smile:

I had already established a connection which was confirmed via the MQTT mosquito broker log:

1598398605: New client connected from 192.168.0.34 as brainESP32 (p2, c1, k15, u'*****').

When I set up my switch as above I could confirm it was being published to the broker via configurations>intergrations>(MQTT)configure. when I typed a ‘#’ in the listen to a topic box and started listening I could see the message being published:

Message 15 received on office/light1/switch at 9:40:
ON
QoS: 0 - Retain: false

Based on @KennethLavrsen advise I had to include client.subscribe("office/light1/switch"); this was missed when following the mqtt_basic example. again thank you, I slotted this in the reconnect loop:

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.println("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect(ID, HA_USER, HA_PASS)) {
      Serial.println("connected");
      //publish test connection
      client.publish("outTopic", "hello world");
      //subscribe to topics
      client.subscribe("office/light1/switch");
    }
    else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      delay(5000);
    }
  }

Additionally, as I had changed my configuration.yaml so much to get it to work I was using the original "/office/light1/switch/" i used to subscribe likewise. Again @KennethLavrsen pointed out it should as per the outgoing message to the broker so after rechecking the configuration.YAML I had to update the Arduino code accordingly by removing the ’ / ’ on either end:

("office/light1/switch") vs ("/office/light1/switch/") 

This resulted in the message passing successfully!!!:

10:20:25.075 -> Topic: office/light1/switch

So now we can read but the read is not executing as expected. therefor I added under Serial.println(topicStr) to check the payload message as per the comparison I was doing below:

Serial.print("payload: ");
Serial.println(payload[0]);

and when i ran it again received on the ardiuno serial monitor:

10:50:13.315 -> Topic: office/light1/switch
10:50:13.315 -> payload: 79

Still, it is not working but this is OK as I’m using a comparison of:
if (payload[0] == 'ON') /if (payload[0] == 'OFF')
and 79 is not ‘ON/OFF’ rather it is the first byte ‘O’ in ASCII. So some slight changes to look at the payload as a string will fix this:

payload[length] = '\0';
String payloadStr = String((char*)payload);

Then change if (payload[0] == 'ON') /if (payload[0] == 'OFF') to if (payloadStr == 'ON') /if (payloadStr == 'OFF')

So at this point, it works, BUT!!! The next step is to incorporate @AaronCake warning and removing the Publishing from the callback as per the PubSubClient and instead set a flag. sure it works, but this could lead to issues later on, so better safe than sorry

This leaves me with:

#include <ArduinoJson.h>
#include <WiFi.h>
#include <PubSubClient.h>
#include <WiFiManager.h>

unsigned long updateCurrentMillis;
unsigned long lastUpdateDelay;
unsigned long updateDelay = 1000;

// MQTT Network
const char* mqtt_server = "192.168.0.200";
//Node ID
const char *ID = "brainESP32";  // Name of our device, must be unique
//Topics
const char *TOPIC = "room/light";  // Topic to subcribe to
const char *STATE_TOPIC = "room/light/state";  // Topic to publish the light state to
int officeLight1Status;

// Home Assistant Credentials
const char *HA_USER = "****";
const char *HA_PASS = "****";

WiFiClient espClient;
PubSubClient client(espClient);

void callback(char* topic, byte* payload, unsigned int length) {
  String topicStr = topic;
  String payloadStr = String(( char *) payload);
  Serial.print("Topic: ");
  Serial.println(topicStr);
  payload[length] = '\0';
  String s = String((char*)payload);
  Serial.println(s);
  if (topicStr == "office/light1/switch") {
    //turn the switch on if the payload is '1' and publish to the MQTT server a confirmation message
    if (s == "ON") {
      Serial.println("light on");
      officeLight1Status = 1;
    }

    //turn the switch off if the payload is '0' and publish to the MQTT server a confirmation message
    else if (s == "OFF") {
      Serial.println("light off");
      officeLight1Status = 0;
    }
  }
}

// Reconnect to client
void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.println("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect(ID, HA_USER, HA_PASS)) {
      Serial.println("connected");
      //publish test connection
      client.publish("outTopic", "hello world");
      //subscribe to topics
      client.subscribe("office/light1/switch");
    }
    else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      delay(5000);
    }
  }
}

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);

  client.setServer(mqtt_server, 1883);
  client.setCallback(callback);
  
  setup_wifi(); // Connect to network
  delay(1500);
}

void setup_wifi() {
  Serial.print("\nConnecting to ");
  //WiFiManager
  WiFiManager wifiManager;
  Serial.println("started");
  //wifiManager.resetSettings(); // turn off after intital settings are done ****** turn on if needing to resetup wifi
  wifiManager.setCustomHeadElement("<style>html{filter: invert(40%); -webkit-filter: invert(40%);}</style>");
  WiFiManagerParameter custom_text("<p>This is just a text paragraph</p>");
  wifiManager.addParameter(&custom_text);
  wifiManager.setAPStaticIPConfig(IPAddress(100, 100, 100, 100), IPAddress(100, 100, 100, 100), IPAddress(255, 255, 255, 0));
  wifiManager.autoConnect();

  while (WiFi.status() != WL_CONNECTED) { // Wait for connection
    delay(500);
    Serial.print(".");
  }
  Serial.println();
  Serial.println("WiFi connected");
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());
}


void loop() {
  if (!client.connected()) {
    reconnect();
  }
  client.loop();
  delay(100);
  if (officeLight1Status != 3)  {
    if (officeLight1Status == 1) {
      client.publish("office/light1/status", "ON");
    }
    else client.publish("office/light1/status", "OFF");
    officeLight1Status = 3;
  }
  updateCurrentMillis = millis(); //send update every x seconds
  if (updateCurrentMillis - lastUpdateDelay >= updateDelay) {
    lastUpdateDelay = updateCurrentMillis;
    switchOut();  //dump data
  }
}

void switchOut() {
  const size_t capacity = JSON_OBJECT_SIZE(26);
  DynamicJsonDocument doc1(capacity);
  DynamicJsonDocument doc2(capacity);
  doc1["pbrAM"] = 0;
  doc1["pbrSS"] = 0;
  doc1["lightAM"] = 0;
  doc1["lp_1"] = 0;
  doc1["lp1_1"] = 0;
  doc1["lp1_2"] = 0;
  doc1["lp1_3"] = 1;
  doc1["lp1_4"] = 0;
  doc1["lp_2"] = 0;
  doc1["lp2_1"] = 1;
  doc1["lp2_2"] = 0;
  doc1["lp2_3"] = 0;
  doc1["lp2_4"] = 0;
  char buffer[256];
  size_t n = serializeJson(doc1, buffer);
  client.publish("brainOut1", buffer, n);
  doc2["tempAM"] = 0;
  doc2["tempSS"] = 1;
  doc2["airAM"] = 0;
  doc2["airSS"] = 0;
  doc2["doseAM"] = 0;
  doc2["doseSS"] = 1;
  doc2["dosephU"] = 0;
  doc2["dosepHD"] = 0;
  doc2["doseNut"] = 0;
  doc2["doseSam"] = 0;
  doc2["doseFill"] = 0;
  doc2["harvsetAM"] = 0;
  doc2["harvestSS"] = 0;
  n = serializeJson(doc2, buffer);
  client.publish("brainOut2", buffer, n);
}

Again thank you for the help guys!

I do not see anywhere that you cannot publish inside a callback function

The issue to watch out for is that the char array and byte array for topic and payload are reused by the publish function.

This means that if your callback function uses topic or payload (which it always do) then you lose your original topic and payload strings the minute you do a publish. They get set to the new topic and payload string

There are two ways to avoid this problem.

Either to ensure that you only use the topic and payload before the publish call and then return out of the function without further testing of the content.

Or you have to create a copy of the topic string and the payload string at the beginning of the callback function and use these copies from then on. Remember it is the entire string that needs to be copied. Not just the pointer to the string.

In most cases where you have a number of “if-else if” tests looking for multiple topic values one at a time only one of them will match. And same with payload. The key is that when you call publish then you kill the topic and payload variable content.

I have some callback functions where I test for a payload value and send a publish as a result. The key is that I have ensured that I leave the callback function after the publish without testing or using the topic and payload strings again.

Yes, it looks like they have changed the documentation possibly when they updated the library. About 6 months ago it had a warning in the API documentation not to publish from a callback, and a specific example in \Examples directory showing how to do this via flag with the same warning. Perhaps the library has been updated to make this less problematic though the documentation and example still warns that the topic and payload variables will be overwritten.

I see what you guys are talking about now :sweat_smile: well that’s handy information, I think for my case I will go ahead publish directly in the callback for the switches while also setting a flag so I can keep track/change elsewhere in my program. The more I’m using MQTT the more I like it!

https://pubsubclient.knolleary.net/api#callback

Internally, the client uses the same buffer for both inbound and outbound messages. After the callback function returns, or if a call to either publish or subscribe is made from within the callback function, the topic and payload values passed to the function will be overwritten. The application should create its own copy of the values if they are required after the callback returns.