ESPhome Sensors only work with API not MQTT

[Rubber9122]

Hi All,
I have a weird situation. I need to make a remote ESPhome device that will be able to communicate only through MQTT but when I only use MQTT most sensors are missing. However all sensors are working fine over API.
Discovery seems to work for only a few diagnostic sensors, but not for custom ones.
This is only with MQTT:

And if I add the device with ESPhome integration:
Some sensors are doubled that are coming from both sources.
And the MQTT side looks fine with all sensor/switch having a config entry for under homeassistant topic:

Here is my ESPhome config, hope that helps:

substitutions:
  name: "esptest1"
  friendly_name: esptest1

esphome:
  name: ${name}
  friendly_name: ${friendly_name}
  min_version: 2024.6.0
  name_add_mac_suffix: false
  project:
    name: esphome.web
    version: '1.0'

esp32:
  board: esp32dev
  framework:
    type: arduino

logger:
  level: VERBOSE

api:
  encryption:
    key: "yeqmICIZpD9Zwt5uhXMim/gt5j2bkoIGEis4oFdh6gA="

ota:
  - platform: esphome
    password: "572acc0dd73a83de15a7cf0502413d5c"

wifi:
  networks:
  - ssid: "ssid0"
    password: !secret wifi_pwd
  - ssid: "ssid1"
    password: !secret wifi_pwd

  ap:
    ssid: "ESPH-test"
    password: "test"

captive_portal:

mqtt:
  broker: IP
  port: 1883
  username: mqtt_esph_user
  password: !secret mqtt_esph_pwd

web_server:

http_request:
  verify_ssl: false

uart:
  rx_pin:
    number: GPIO16
  baud_rate: 115200
  rx_buffer_size: 3000

dsmr:
  crc_check: false
  max_telegram_length: 3000

sensor:
  - platform: dsmr
    energy_delivered_lux:
      name: Energy Consumed
  - platform: wifi_signal # Reports the WiFi signal strength/RSSI in dB
    name: "WiFi Signal dB"
    id: wifi_signal_db
    update_interval: 60s
    entity_category: "diagnostic"
  - platform: copy # Reports the WiFi signal strength in %
    source_id: wifi_signal_db
    name: "WiFi Signal Percent"
    filters:
      - lambda: return min(max(2 * (x + 100.0), 0.0), 100.0);
    unit_of_measurement: "Signal %"
    entity_category: "diagnostic"
    device_class: ""
  - platform: template
    name: "Module Version"
    id: module_version
    entity_category: "diagnostic"
    lambda: |-
      return 0;
  - platform: uptime
    type: seconds
    update_interval: 60s
    name: Uptime
    unit_of_measurement: "s"
    device_class: "duration"
    entity_category: "diagnostic"

text_sensor:
  - platform: dsmr
    identification:
      name: "DSMR Identification"
    p1_version:
      name: "DSMR Version"
    electricity_tariff:
      name: "Electric Tariff"
  - platform: version
    name: "ESPHome Version"
    hide_timestamp: true

switch:
  - platform: restart
    name: ESP restart

I have already tried multiple versions of ESPhome and HA

Thank for the help in advance!

[Rubber9122]