MQTT sensor issues

Aephir · June 3, 2018, 11:01pm

Hi,

I have “slave” hassio running on a RPi3B reading some plant sensors. I’m connecting the hassio to mosquitto running on NUC that is also running the “master” instance of home assistant. hassio is using the mosquitto running on the NUC.

mqtt_publish using an automation on the hassio (for a Mi Plant sensor) works fine; automation.yaml on hassio:

- alias: plant sensor 1 send state
  trigger:
    platform: time
    seconds: 30 # triggers every 30 seconds
  action:
    - service: mqtt.publish
      data:
        topic: plant_sensor/1
        payload_template: "{ \"state\": {{ states('plant.basil') }}, \"battery\": {{ states('sensor.plant_sensor_battery') }}, \"temperature\": {{ states('sensor.plant_sensor_temperature') }}, \"conductivity\": {{ states('sensor.plant_sensor_conductivity') }}, \"light_intensity\": {{ states('sensor.plant_sensor_light_intensity') }}, \"moisture\": {{ states('sensor.plant_sensor_moisture') }} }"

Using mosquitto_sub -h 127.0.0.1 -v -t "plant_sensor/#" in the NUC terminal, I can see the data arriving on the NUC:

plant_sensor/1 { "state": problem, "battery": 99, "temperature": 20.0, "conductivity": 336, "light_intensity": 73, "moisture": 28 }

But I can’t seem to get the mqtt sensors working on the NUC. In the sensor.yaml, I have:

- platform: mqtt
  name: mi_sensor_1_state
  state_topic: plant_sensor/1
  value_template: "{{ value_json.state }}"
- platform: mqtt
  name: mi_sensor_1_battery
  state_topic: plant_sensor/1
  unit_of_measurement: '%'
  value_template: "{{ value_json.battery }}"
  device_class: battery
- platform: mqtt
  name: mi_sensor_1_temperature
  state_topic: plant_sensor/1
  unit_of_measurement: '°C'
  value_template: "{{ value_json.temperature }}"
  device_class: temperature
- platform: mqtt
  name: mi_sensor_1_conductivity
  state_topic: plant_sensor/1
  unit_of_measurement: 'µS/cm'
  value_template: "{{ value_json.conductivity }}"
  device_class: temperature
- platform: mqtt
  name: mi_sensor_1_light_intensity
  state_topic: plant_sensor/1
  unit_of_measurement: 'lx'
  value_template: "{{ value_json.light_intensity }}"
  device_class: temperature
- platform: mqtt
  name: mi_sensor_1_moisture
  state_topic: plant_sensor/1
  value_template: "{{ value_json.moisture }}"
  device_class: humidity

But they don’t update. The states of the mqtt sensors stay as “unknown”.

Any idea what I’m missing?

ReneTode · June 3, 2018, 11:05pm

did you configure mqtt right on the nuke HA?

Aephir · June 4, 2018, 7:06am

I should think so. I’m using this script to get location and battery updates from our phones. It uses mqtt, and works fine in home assistant on the NUC.

mosquitto.yaml

broker: 127.0.0.1
port: 1883
client_id: homeassistant_mqtt
username: !secret mosquitto_user
password: !secret mosquitto_pass
protocol: 3.1

with a very minimal mosquitto.conf:

pid_file /var/run/mosquitto.pid

persistence true
persistence_location /mqtt/data/

user mosquitto

port 1883

log_dest file /mqtt/log/mosquitto.log
log_dest stdout

and the docker-compose.yaml:

version: '3'
services:
  mqtt:
    image: eclipse-mosquitto:latest
    container_name: "MQTT"
    restart: always
    ports:
      - "1883:1883"
      - "9001:9001"
    volumes:
      - /home/aephir/docker/mosquitto/data:/mqtt/data
      - /home/aephir/docker/mosquitto/config:/mqtt/config:ro
      - /home/aephir/docker/mosquitto/log:/mqtt/log
    network_mode: "host"

gpbenton · June 4, 2018, 7:48am

That looks fine to me, although with network_mode: "host" you do not need to specify open ports. But that is not your problem.

I would first try putting quotes around the state_topic: - I think the / may be confusing the yaml.

If that doesn’t work, you could add mqtt debug to the logging component and looking in the ha log to see if it is actually seeing the incoming message.

Aephir · June 4, 2018, 9:21am

I’ve put homeassistant.components.mqtt: notset in the logger.yaml, and see repeatedly:

2018-06-04 09:01:41 INFO (Thread-2) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
2018-06-04 09:01:42 DEBUG (MainThread) [homeassistant.components.mqtt] Subscribing to dummy/floorplan/sensor
2018-06-04 09:01:42 DEBUG (MainThread) [homeassistant.components.mqtt] Subscribing to owntracks/#
2018-06-04 09:01:42 DEBUG (MainThread) [homeassistant.components.mqtt] Subscribing to owntracks/Emilie/SECRET_ID
2018-06-04 09:01:42 DEBUG (MainThread) [homeassistant.components.mqtt] Subscribing to plant_sensor/1
2018-06-04 09:01:43 DEBUG (MainThread) [homeassistant.components.mqtt] Received message on owntracks/Emilie/SECRET_ID: b'{"t":"p","tst":NUMBER,"acc":50,"_type":"location","alt":0,"lon":SECRET_LON,"lat":SECRET_LAT,"batt": 1}'

So the life360 stuff comes through (and updates in home assistant on NUC), but the messages from the RPi hassio does not. They all show up if I subscribe on NUC terminal, but not in home assistant on the NUC.

For comparison, the battery stat sensors for the life360 script looks like this in sensor.yaml:

- platform: mqtt
  state_topic: "owntracks/Emilie/SECRET_ID"
  name: "Emilie Phone Battery 360"
  unit_of_measurement: "%"
  value_template: "{{ value_json.batt }}"

Also, I now have state_topic in quotes (“”), but no luck. It works, just not for these specific sensors

gpbenton · June 4, 2018, 10:10am

This should only happen once at startup. How oftend does it repeat?

Aephir · June 4, 2018, 10:19am

Hmmm… The

2018-06-04 10:14:15 INFO (Thread-2) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server

repeats every 1-2 seconds… Soo… something’s definitely wrong…?

I found this issue, so I removed the container and image, and re-downloaded it. Again, no luck.

gpbenton · June 4, 2018, 11:02am

The most common reason for this I have seen is two clients with the same client_id:. Each client (including each HA instance) has to have a unique id.

Edit:
The mosquitto log file will normally tell you why it is disconnecting a client.

Aephir · June 5, 2018, 10:48am

It works!

The problem with re-connecting (that I didn’t know I had) was solved by removing the client_id from mqtt.yaml (it didn’t work to give different client_id on RPi and NUC. I had to remove the client_id line altogether). After doing that, the sensor data showed up in the homeassistant.log (but wasn’t read by homeassistant mqtt sensor).

Adding \" around the template value (for state only, maybe because this one is not a number?) makes home assistant (on NUC) recognize the whole json string. So the automation.yaml on RPi needed to be:

- alias: plant sensor 1 send state
  trigger:
    platform: time
    seconds: 30 # triggers every 30 seconds
  action:
    - service: mqtt.publish
      data:
        topic: plant_sensor/1
#        payload_template: "{ \"state\": {{ states('plant.basil') }}, \"battery\": {{ states('sensor.plant_sensor_battery') }}, \"temperature\": {{ states('sensor.plant_sensor_temperature') }}, \"conductivity\": {{ states('sensor.plant_sensor_conductivity') }}, \"light_intensity\": {{ states('sensor.plant_sensor_light_intensity') }}, \"moisture\": {{ states('sensor.plant_sensor_moisture') }} }"
        payload_template: "{ \"state\": \"{{ states('plant.basil') }}\", \"battery\": {{ states('sensor.plant_sensor_battery') }}, \"temperature\": {{ states('sensor.plant_sensor_temperature') }}, \"conductivity\": {{ states('sensor.plant_sensor_conductivity') }}, \"light_intensity\": {{ states('sensor.plant_sensor_light_intensity') }}, \"moisture\": {{ states('sensor.plant_sensor_moisture') }} }"

Compare to the old line (commented out). The message received is now:

plant_sensor/1 { "state": "problem", "battery": 99, "temperature": 32.4, "conductivity": 765, "light_intensity": 112630, "moisture": 45 }

The mqtt.yaml is very minimal; on NUC:

broker: 127.0.0.1
port: 1883
protocol: 3.1

and on the RPi:

broker: 192.168.0.100
port: 1883
protocol: 3.1

So thanks @ @gpbenton, I think I’ve solved the problems, including the one I didn’t even know I had

And btw, I had some old logs in the mosquitto.log (from before removing client_id). I don’t think these are very helpful, or important anymore?:

1527844867: mosquitto version 1.4.12 (build date 2017-06-01 13:03:46+0000) starting
1527844867: Config loaded from /mosquitto/config/mosquitto.conf.
1527844867: Opening ipv4 listen socket on port 1883.
1527844867: Opening ipv6 listen socket on port 1883.
1527844876: New connection from 127.0.0.1 on port 1883.
1527845021: mosquitto version 1.4.12 (build date 2017-06-01 13:03:46+0000) starting
1527845021: Config loaded from /mosquitto/config/mosquitto.conf.
1527845021: Opening ipv4 listen socket on port 1883.
1527845021: Opening ipv6 listen socket on port 1883.
1527845029: New connection from 127.0.0.1 on port 1883.
1527845029: New client connected from 127.0.0.1 as homeassistant_mqtt (c1, k60, u'homeassistant').
1527846821: Saving in-memory database to /mosquitto/data/mosquitto.db.
1527848622: Saving in-memory database to /mosquitto/data/mosquitto.db.
1527850423: Saving in-memory database to /mosquitto/data/mosquitto.db.
1527851733: Client homeassistant_mqtt disconnected.
1527851733: Error in poll: Interrupted system call.
1527851733: mosquitto version 1.4.12 terminating
1527854236: mosquitto version 1.4.12 (build date 2017-06-01 13:03:46+0000) starting
1527854236: Config loaded from /mosquitto/config/mosquitto.conf.
1527854236: Opening ipv4 listen socket on port 1883.
1527854236: Opening ipv6 listen socket on port 1883.
1527854243: New connection from 127.0.0.1 on port 1883.
1527854243: New client connected from 127.0.0.1 as homeassistant_mqtt (c1, k60, u'homeassistant').
1527854852: Error in poll: Interrupted system call.
1527854852: mosquitto version 1.4.12 terminating
1527854945: mosquitto version 1.4.12 (build date 2017-06-01 13:03:46+0000) starting
1527854945: Config loaded from /mosquitto/config/mosquitto.conf.
1527854945: Opening ipv4 listen socket on port 1883.
1527854945: Opening ipv6 listen socket on port 1883.
1527854952: New connection from 127.0.0.1 on port 1883.
1527854990: mosquitto version 1.4.12 (build date 2017-06-01 13:03:46+0000) starting
1527854990: Config loaded from /mosquitto/config/mosquitto.conf.
1527854990: Opening ipv4 listen socket on port 1883.
1527854990: Opening ipv6 listen socket on port 1883.
1527855000: New connection from 127.0.0.1 on port 1883.
1527855000: New client connected from 127.0.0.1 as homeassistant_mqtt (c1, k60, u'homeassistant').
1527855683: Client homeassistant_mqtt disconnected.
1527855683: Error in poll: Interrupted system call.
1527855683: mosquitto version 1.4.12 terminating
1527855901: mosquitto version 1.4.12 (build date 2017-06-01 13:03:46+0000) starting
1527855901: Config loaded from /mosquitto/config/mosquitto.conf.
1527855901: Opening ipv4 listen socket on port 1883.
1527855901: Opening ipv6 listen socket on port 1883.
1527855907: New connection from 127.0.0.1 on port 1883.

etc., but I get no new entries upon reboot. As in zero new lines; I deleted all the content of the file, now it’s empty even after multiple reboots.

gpbenton · June 5, 2018, 1:28pm

The last line in your log file was from 1st June

$ date --date="@1527855907"
Fri  1 Jun 13:25:07 BST 2018

I suspect that only the second log_dest line in your config file had any effect, so mosquitto logs are only going to stdout, but I have never tried two entries like that so I’m not sure.