Entities duplicated

Hi all,

I am preparing an ESP board and configuration. I have included a MQTT sensor and noticed that my entities are being duplicated. I am not sure if this is the result of the MQTT configuration or just a co-incidence.

This is the issue (sensors are currently offline, but you get the idea):

And this is my Esp configuration:

esphome:
  name: boiler1

esp32:
  board: esp32-s3-devkitc-1
  framework:
    type: esp-idf

# Enable logging
logger:

# Enable Home Assistant API
api:

# Allow Over-The-Air updates
ota:
- platform: esphome

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  id: wifi_component

web_server:
  port: 80

# Enable MQTT
mqtt:
  broker: 192.168.1.12  # Replace with your MQTT broker IP
  username: secret :)
  password: secret :)
  discovery: false
  
uart:
  id: uart_bus
  baud_rate: 9600
  tx_pin: GPIO43
  rx_pin: GPIO44
  debug:
    direction: RX
    dummy_receiver: true
    after:
      delimiter: "\r\n"
    sequence:
      - lambda: |-
          //UARTDebug::log_string(direction, bytes);  //Still log the data
          
          int sensorTEMP=0; 
          
          //Example to convert uart text to string
          std::string str(bytes.begin(), bytes.end());

          //watch for potential problems with non printable or special characters in string
          //id(rawString).publish_state(str.c_str());
          
          //Sample uart text protocol that sends id:temperature\r\n packets
          if (sscanf(str.c_str(), "T:%d", &sensorTEMP) == 1 ) {
            id(boiler_temp).publish_state(sensorTEMP); 
          }

sensor:
  - platform: template
    name: "Temperature"
    id: boiler_temp
    unit_of_measurement: "°C"
    accuracy_decimals: 0
  - platform: mqtt_subscribe
    name: "Surplus power"
    id: pwr_setpoint
    topic: ess/surplus_setpoint
    accuracy_decimals: 0
    unit_of_measurement: "W"


interval:
  - interval: 1s
    then:
      - uart.write: "GET_TEMP\r\n"

I have tried to remove the device, rebooted HA, added the device (even under a new name) but the same condition reappears. Removing the entitities I don’t want is not possible.

What am I doing wrong?

You enabled both the HA API and MQTT, so the esp is using two separate methods to communicate the entities. So I’m not really surprised you have everything twice.

Right, thanks.

But I believe the HA Api is preferred, I just want to listen to a specific MQTT topic without exposing it’s entities.

What would be the preferred method?

Preferred for what?

Listen for what from where and why?

I hear many people say the HA api stuff is better, but I don’t believe I have ever heard exactly how it is better.

I have been using MQTT for more than a decade and it has been incredibly reliable, but I generally do NOT use retained messages. HA appears to have a strong reliance on retained messages, which are incredibly easy to mess up, which then makes things incredibly confusing.

My guess is they came up with the API to eliminate the dependency on retained messages (since they can just go query each device whenever they want). If you are only using HA, use the API, it likely is better.

I use much more than HA and HA is just a tiny offshoot in my system, so the interoperability of MQTT is incredibly important to me. I actually turn off the API on my esphome devices.

Not really that difficult.

State or discovery message → retain
Command message → do not retain

However I use the API for the reasons listed here: Native API Component — ESPHome

mqtt:
  broker: 192.168.0.99
  username: xxxx
  password: !secret wifi_password
  topic_prefix: null # Use null to disable publishing or subscribing of any MQTT topic unless it is explicitly configured.

Yet people (other than you) have been struggling with it for years. I remember the pain many people felt during the early days of HA trying to use it with Tasmota and getting stuff not showing up or showing up stuff that had long been deleted/turned off. I still see many people struggling with devices disappearing or ghost devices that won’t go away.

As far as the benefits of the HA API go, if you ONLY use HA I am sure they provide some benefit. I don’t really use HA. I am on my third attempt, but the cost of learning it has been higher than the benefit it provides for my needs. I use MQTT, Node-RED, InfluxDB, Grafana. It does NOT do automatic discovery, but it also doesn’t screw that up, ever :wink:.

Don’t get me wrong I use esphome and think it is good and it makes some things really easy, same for HA. Other things take more work.

Ha! Yeah that was definitely a thing. I remember posting The Hookup’s youtube video many times as an explanation of why their Tasmota devices were “ghost” switching.

Huh, I would not have picked that. Especially as you remember the early days.

Anyway getting a bit off-topic.

Thanks for that ! I added MQTT to tell my greenhouse ESP32 not to return to deep_sleep when i want to make a change or there is an OTA firmware update (which works well), but all my other entities got duplicated - which I wasn’t expecting.

Thanks all!

I am however still not able to get rid of the duplicate entities and I am not sure what to do now.

First, I have changed my config to this:

# Enable MQTT
mqtt:
  broker: 192.168.1.12  # Replace with your MQTT broker IP
  username: secret :)
  password: secret :)
  discovery: false
  topic_prefix: null # Use null to disable publishing or subscribing of any MQTT topic unless it is explicitly configured.
  

And flashed by device.

Then, I have removed all the (retained) topics for this device in MQTT explorer. Basically, I have removed all traces to this device to be sure.

Then, I have removed the device from ESPhome and rebooted Home Assistant.

The device is discovered in ESPhome devices overview, but again with the duplicated entities as mentioned in the startpost.

The auto discovery topics are in “homeassistant” (default).
So you have a topic for auto discovery and another with the sensor values.
You need to remove them too.

When I’m doing a test, I usually select the IP in the MQTT explorer and delete everything, letting MQTT rebuild itself again.
Be careful with this procedure, because a device that sends few updates will disappear from HA until the next update.

Ok, will test that!

This fixed it !