MQTT state stream to AWS IoT

PtP · June 1, 2018, 1:59pm

I am attempting to send updated states of all my sensors to AWS IoT and then a DB like DynamoDB.

I have My MQTT set up as so…

mqtt:
  broker: asdvuwwnvwiev.iot.us-west-2.amazonaws.com
  port: 8883
  client_id: piGateway
  certificate: /home/homeassistant/.homeassistant/AWSIoT/root-CA.crt
  client_key: /home/homeassistant/.homeassistant/AWSIoT/piGateway.private.key
  client_cert: /home/homeassistant/.homeassistant/AWSIoT/piGateway.cert.pem
  protocol: 3.1.1
  tls_version: '1.2'

mqtt_statestream:
  base_topic: homeassistant

With this running I get a constant stream in AWS IoT

Here is what my log looks like:

Jun 01 08:37:13 proto-gateway hass[730]: 2018-06-01 08:37:13 INFO (Thread-10) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
Jun 01 08:37:15 proto-gateway hass[730]: 2018-06-01 08:37:15 INFO (Thread-10) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
Jun 01 08:37:16 proto-gateway hass[730]: 2018-06-01 08:37:16 INFO (Thread-10) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
Jun 01 08:37:17 proto-gateway hass[730]: 2018-06-01 08:37:17 INFO (Thread-10) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
Jun 01 08:37:19 proto-gateway hass[730]: 2018-06-01 08:37:19 INFO (Thread-10) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
Jun 01 08:37:20 proto-gateway hass[730]: 2018-06-01 08:37:20 INFO (Thread-10) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
Jun 01 08:37:21 proto-gateway hass[730]: 2018-06-01 08:37:21 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: new_state=<state sensor.memory_use=123.3; friendly_name=Memory use, unit_of_measurement=MiB, icon=mdi:memory @ 2018-06-01T13:37:21.709924+00:00>, old_state=<state sensor.memory_use=123.2; friendly_name=Memory use, unit_of_measurement=MiB, icon=mdi:memory @ 2018-06-01T13:36:50.706864+00:00>, entity_id=sensor.memory_use>
Jun 01 08:37:21 proto-gateway hass[730]: 2018-06-01 08:37:21 INFO (MainThread) [homeassistant.core] Bus:Handling <Event call_service[L]: domain=mqtt, service=publish, service_data=retain=True, topic=homeassistant/sensor/memory_use/state, qos=1, payload=123.3, service_call_id=1975993776-20>
Jun 01 08:37:21 proto-gateway hass[730]: 2018-06-01 08:37:21 DEBUG (MainThread) [homeassistant.components.websocket_api] WS 1760767568: Sending {'type': 'event', 'event': {'origin': 'LOCAL', 'event_type': 'state_changed', 'time_fired': datetime.datetime(2018, 6, 1, 13, 37, 21, 710080, tzinfo=<UTC>), 'data': {'new_state': <state sensor.memory_use=123.3; friendly_name=Memory use, unit_of_measurement=MiB, icon=mdi:memory @ 2018-06-01T13:37:21.709924+00:00>, 'old_state': <state sensor.memory_use=123.2; friendly_name=Memory use, unit_of_measurement=MiB, icon=mdi:memory @ 2018-06-01T13:36:50.706864+00:00>, 'entity_id': 'sensor.memory_use'}}, 'id': 2}
Jun 01 08:37:21 proto-gateway hass[730]: 2018-06-01 08:37:21 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: new_state=<state sensor.processor_use=3; friendly_name=Processor use, unit_of_measurement=%, icon=mdi:memory @ 2018-06-01T13:37:21.742752+00:00>, old_state=<state sensor.processor_use=2; friendly_name=Processor use, unit_of_measurement=%, icon=mdi:memory @ 2018-06-01T13:36:50.724453+00:00>, entity_id=sensor.processor_use>
Jun 01 08:37:21 proto-gateway hass[730]: 2018-06-01 08:37:21 INFO (MainThread) [homeassistant.core] Bus:Handling <Event service_executed[L]: service_call_id=1975993776-20>
Jun 01 08:37:21 proto-gateway hass[730]: 2018-06-01 08:37:21 INFO (MainThread) [homeassistant.core] Bus:Handling <Event call_service[L]: domain=mqtt, service=publish, service_data=retain=True, topic=homeassistant/sensor/processor_use/state, qos=1, payload=3, service_call_id=1975993776-21>
Jun 01 08:37:21 proto-gateway hass[730]: 2018-06-01 08:37:21 DEBUG (MainThread) [homeassistant.components.websocket_api] WS 1760767568: Sending {'type': 'event', 'event': {'origin': 'LOCAL', 'event_type': 'state_changed', 'time_fired': datetime.datetime(2018, 6, 1, 13, 37, 21, 742904, tzinfo=<UTC>), 'data': {'new_state': <state sensor.processor_use=3; friendly_name=Processor use, unit_of_measurement=%, icon=mdi:memory @ 2018-06-01T13:37:21.742752+00:00>, 'old_state': <state sensor.processor_use=2; friendly_name=Processor use, unit_of_measurement=%, icon=mdi:memory @ 2018-06-01T13:36:50.724453+00:00>, 'entity_id': 'sensor.processor_use'}}, 'id': 2}
Jun 01 08:37:21 proto-gateway hass[730]: 2018-06-01 08:37:21 INFO (MainThread) [homeassistant.core] Bus:Handling <Event service_executed[L]: service_call_id=1975993776-21>
Jun 01 08:37:22 proto-gateway hass[730]: 2018-06-01 08:37:22 INFO (Thread-10) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
Jun 01 08:37:23 proto-gateway hass[730]: 2018-06-01 08:37:23 INFO (Thread-10) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
Jun 01 08:37:25 proto-gateway hass[730]: 2018-06-01 08:37:25 INFO (Thread-10) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
Jun 01 08:37:26 proto-gateway hass[730]: 2018-06-01 08:37:26 INFO (Thread-10) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server

If I subscribe to the topic nothing comes in:

Alternatively if I stop MQTT state stream and manually publish a topic from HA and subscribe to it in AWS it comes through.

Is it possible to do the state stream to AWS IoT?

Do I have to manually set up automations for each sensor I have?

The second item I need to accomplish is to get it to an organized db.
I tried setting up a rule for my Thing but nothing seemed to work.

If there are any posts that I have over looked please let me know.

Otherwise I appreciate any input on this topic.

Thanks!

PtP · June 5, 2018, 12:55pm

I found this entry on another post which will accomplish what I need on the MQTT side.
Thanks @NotoriousBDG for that input!

Now I am just learning how to make a table with the json sent to my “Thing”. The response that triggers my “Rule” looks like this.

{
  "device_class": "sensor",
  "name": "Avg Lasers",
  "distance": "5.859",
  "min_value": "5.797",
  "max_value": "5.922",
  "unit_of_measurement": "feet"
}

If anyone has some DynamoDB table building pointers I am all ears.

Thanks!

PtP · June 5, 2018, 7:15pm

I think I have it all figured out now.
I needed to send a generic value such as “hub_id” in my MQTT message and then use that as the Partition Key, Then I could use the “name” as a sort key.