Handling large MQTT Payloads

ESPHome
1

Hello everyone,

I have a epaper display where I want to show many information depending on what’s currently going on.

Currently I try to rebuild it using a service on a external device, for generating images, compressing them (between 8000 - 10000 bytes) and publish them to MQTT.

The display is connected to an esp with ESPHome. But unfortunately I discovered that the esp crashes when receiving more than about 4000 characters over MQTT, which is kind of strange because my esp have a RAM of 512 kb, which should be free enough, especially since ESPHome says there is only a usage of 8%.

Just for mentioning, I’m using the MqttClient.subscribe method and the crash also appear on only logging that a message has been received.

Do you have an idea how I can increase the limit or should I prefere the HTTP client for dealing with that large payloads?

Thank you for your advice

2

I was searching around for a while and discover that there is a buffer for MQTT Messages with a length of 4096 bytes, which explains the crash.

I switched to esp-idf with the following config:

esp32:
  board: az-delivery-devkit-v4
  framework:
     type: esp-idf
     version: recommended
     sdkconfig_options:
       COMPILER_OPTIMIZATION_SIZE: y
       CONFIG_MQTT_USE_CUSTOM_CONFIG: y
       CONFIG_MQTT_BUFFER_SIZE: "16384" # 2^14
     advanced:
       ignore_efuse_mac_crc: false

But it also crash when receiving large payloads.

3

Instead of storing the compressed image (in base64, I’m assuming), can your external service save the compressed image and then send a pointer to that image uri in the payload?

4

That’s sound also like an interesting idea.
Currently I’m simply cast the sting data to a unsigned short array.
Can I use the http client to get directly binary data?

5

I don’t see why you couldn’t as long as it’s being saved at a valid URI. So, store the URI in MQTT, then use the client to actually grab the binary.

Seems cleaner to me. :blush: