Bulit-in MQTT broker

yesimwilliam · January 28, 2018, 8:17pm

Can I use the built-in MQTT broker as-is to track my phone running owntracks app?

If so what do I need to set up to get this to work? I’m looking through the docs and seem to be going in circles trying to figure it out.

Any help would be appreciated.

Thanks

olfreu · January 28, 2018, 10:42pm

Actually you can do this yes. Seems as if the owntracks supports mqtt http://owntracks.org/booklet/tech/mqtt/. So you “just” have to make the HA-Instance and the port 1883 (if default) accessable through the internet, like with duckdns and portforwarding, and then push your location with the build in api of owntracks. Just set the broker address to your duckdns-domain and the port to 1883 (if default) and then use the yaml to read the topic out with your HA.

yesimwilliam · January 29, 2018, 1:51am

How do I set the broker address? I have never used mqtt so I’m a total noob with this. So could you dumb it down for me.

So how do I set the broker address to my DDNS.
What do you mean “read the topic out”?

HeffeD · January 29, 2018, 3:21am

I was also new to MQTT and just couldn’t figure out how to use the embedded broker. Like so many things with Home Assistant, the documentation always seems to assume you have some previous knowledge on the subject and seem to be missing key bits of information… I ended up using the OwnTracks (via HTTP) component. It was a much more straightforward setup for me.

olfreu · January 29, 2018, 12:15pm

Well you can use the integrated mosquitto plugin of home assistant and just start the mqtt server. The mqtt-protocol is based on “topics”, for example there is a topic called “geolocation/mobile_device_xy/location” (depends on what you want to call it, no format given, “geolocation” would also work but then all devices will push their information in the same channel). Now you tell the HA to watch (called subscription) this topic. Every information published inside of this topic will get noticed by the HA. Same from the “client” side. You tell the OwnTracks to publish the information of its location to the same topic. Now if the client publishes the information, the server/HA notices it and triggers the desired command. If you can’t imagine that why, just imagine it like a chat channel (btw is mostly also based on websockets which is similar to mqtt). 2 people are in there. One is the server and one the client. Now if the first person writes, the second one get’s a notification and the other way around. So the server can also publish data to the channel, but the client only get’s that, if he also subscribed the topic. In other words:

subscription = receive infos from this channel (in this just the server and if needed for qos also the client)
publish = emit to the channel (in this case the client, your mobile phone or OwnTracks and if needed for the qos also the client)

So as you noticed, there are information in the channel but you can also define the flow of these with qos, means should it be pushed and the receiver has to answer to validate the publish or should it just be pushed and thats it, no guarantee of receiving by the subscribers but less traffic.

Ajax/HTTP is nice but not as quick and dynamic as websockets or mqtt. I prefer these over ajax, because of real time sync of data between multiple components. In this case it will also do its thing, but I personally wouldn’t use it if the frequency of the data submits are too high or if it has to be reliable, then I would use mqtt/websockets (https://www.peterbe.com/plog/are-websockets-faster-than-ajax or https://stackoverflow.com/questions/33793872/performance-of-ajax-vs-websocket-rest-over-http-2-0).

The broker address is just the port 1883 and your normal HA duckdns domain. So you just have to forward the port 1883 with your router, then it should work.

HeffeD · January 29, 2018, 5:55pm

Thank you for your input!

olfreu · January 29, 2018, 9:01pm

You are welcome! Hope this helped a bit

yesimwilliam · January 30, 2018, 12:49am

It’s getting a little clearer now, thank you for explanation.

So I added mqtt to my config file, thats all I did in the config. I’m sure there is something else I need to add. I forwarded 1883 port and added my DDNS address in the owntracks app.

It shows connected in the app.

Back on the HA it shows my phone as a badge at the top and its showing the correct GPS coordinnates, show battery level and so forth so I know its working, but how to I get it to show up on the map. Also I still don’t know where to put the topics in the config file or the app on my phone.

Thanks

olfreu · January 30, 2018, 12:37pm

Well as sensor you need then something like this in the configuration.yaml:

sensor 10:
  -platform: mqtt
   name: "My phone"
   state_topic: "track/phone/location" #here is the topic which you push by the owntracks app
   qos: 0

To use it with the map and not just as sensor, you have to take a look at this link https://home-assistant.io/components/device_tracker/. There is a reference to the mqtt part.

In owntracks I think you have to use MQTT Private in the settings and go to the configurations. There is a “pubTopicBase”, which could be the track/phone/location for example. This is the general topic where all publishes will be pushed in and then there is the subTopic, which would be for example track/phone/answer (if you use the same topic as in pubTopicBase, then the subTopic will receive the own pushed data too, so dont use the same path. You can also set sub: false if you dont need the qos>0, so if you dont need to receive anything or get answers from the server). Btw seems as if the owntracks uses 8883 by default, so you have to change that to 1883 if you are using this port. From the mqtt plugin of HA:

mqtt:
  # Optional, replace port 1883 with following if you want encryption
  # (doesn't really matter because broker is public)
  port: 8883

So just use 1883 for both.

yesimwilliam · January 30, 2018, 3:09pm

So when I add this to my yaml config I get all kinds of errors. If you want me to post the errors I will but there is a lot of them.

sensor:
  - platform: mqtt
    name: "My Phone"
    state_topic: "owntracks/william/phone"
    qos: 0

This is what the section looks like,

mqtt:

device_tracker:
  - platform: owntracks
    max_gps_accuracy: 200
 
sensor:
  - platform: mqtt
    name: "Dad's Phone"
    state_topic: "owntracks/william/phone"
    qos: 0

any Idea whats wrong here.

Thanks

techwithjake · January 30, 2018, 3:42pm

Check out how I use(d) Owntracks with Home Assistant.
I don’t use Hass.io, just the virtual but still using Mosquitto.

github.com

techwithjake/home-assistant-config/blob/master/device_trackers.yaml

### Own Tracks ###
#  - platform: owntracks
#    waypoints: False
#    waypoint_whitelist:
#      - xl
#      - dietpi
#    secret: !secret ot_secret

### Bluetooth ###
#  - platform: bluetooth_le_tracker

### NMap ###
  - platform: nmap_tracker
    hosts:
      - 192.168.86.XXX
      - 192.168.86.XXX
      - 192.168.86.XXX
      - 192.168.86.XXX
      - 192.168.86.XXX
      - 192.168.86.XXX

This file has been truncated. show original

I can post up screenshots of my Owntracks as well if you’d like.

olfreu · January 30, 2018, 6:27pm

Well to use the owntrack via mqtt you need to do the following I think:
Go to settings in owntrack, set it to mqtt private.
Set the host to your duckdns domain and the port to 1883.
Set the username and password, maybe the device and tracker id but I think you dont need it.

Then in your config use this for example:

device_tracker:
  - platform: mqtt #or mqtt_json
    devices:
      mydevice: 'owntracks/...'

Check this https://home-assistant.io/components/device_tracker.mqtt/ or this https://home-assistant.io/components/device_tracker.mqtt_json/ for more infos about that. Then it should work and you are reading the device out of the mqtt.

@techwithjake yes but you are not reading the devices out of the mqtt. You are using the direct connection and nmap.

techwithjake · January 30, 2018, 6:36pm

I am using MQTT. That’s what the original Owntracks Integration called for: https://home-assistant.io/components/device_tracker.owntracks/

Settings inside the Owntracks App are:
Private MQTT
https://my.domain.com
8883
My MQTT Auth Details
Device ID
Tracker ID
Installed my Cert from Owntracks Tools
Encryption Key under Advanced with libsodium18 installed on RPi

I’m using Zanzito as a replacement since it has a lot of other abilities to it. One I personally like is switching to a local host when on the same network since my router is screwing up NATing. With Zanzito, it just shows up in my “known_devices.yaml” file.

olfreu · January 30, 2018, 6:44pm

Ah then its even easier to set up yeah.

yesimwilliam · January 30, 2018, 7:46pm

Are all those entries in the yaml file necessary to get this to work correctly?

The strange thing is that my daughters phone shows up on the HA map but not mine. Also since I came to work my HA will not update my phone location even though owntracks has updated to my current location.
I have put no extra code in the HA config file except for what I listed earlier and somehow its works for her phone?

yesimwilliam · February 1, 2018, 5:26am

So I’m beginning to think there is a different setup for Hass.io. Nothing I do seems to work. When It does work it eventually stops working. For example now I cannot get the owntracks app to connect at all. I have changed nothing, so Im’ not sure what the issue is now.

Under Status in the setting portion of the app it says,

idle The operation couldn’t be completed. (kCFErrorDomainCFNetwork error 1.) {

    "_kCFStreamErrorCodeKey" = 1;

    "_kCFStreamErrorDomainKey" = 12;

}

No idea why this is happening.
I did install DDNS with encryption so would that have anything to do with it?

Thanks,
William

sparkydave · February 1, 2018, 7:00am

I have never been able to get OwnTracks working and I dont know why. I have port 1883 forwarded, I have mqtt: in my config and I have

  - platform: owntracks
    max_gps_accuracy: 200

in my device_tracker.yaml yet OwnTracks always says it has an error connecting. My OwnTracks config has my duckdns address, port 1883 and ‘homeassistant’ as username, and API password entered.

any ideas of how to get it to connect?

yesimwilliam · February 2, 2018, 3:03am

So I’m using Hass.io and I installed the mosquitto addon and added a username and password to the addon confid section. Then I duplicated those credentials on the owntracks app.
When I did this I got an invalid confg error on the HA main page

So I added to the config

mqtt: broker: 192.168.1.23
which points to the HA IP address and get ride of the invalid config error.

So it connects fine now. I can actually see in the mosquitto add-on log that it is connected.
So this is what I added to the config file after I enabled the mosquitto add-on.

mqtt:
    broker: 192.168.1.23
  
device_tracker:
  - platform: owntracks
    max_gps_accuracy: 200
    waypoints: True
    mqtt_topic: "owntracks/#"
    events_only: True

I thought this would be all I needed but…my phone badge would not show up though in HA…
How do I get the badge to show up in HA?
If I add another device tracker it gives me errors I guess because you cannot have two device tracker at the same time? I did fiddle around and got the badge to show up but the problem was after I edited the entry in known_devices and rebooted HA it would duplicate the same device in this case my phone.

So I definitely have something confidured wrong…

Any thoughts…

So I added

sparkydave · February 2, 2018, 3:50am

Since I dont yet have OwnTracks working, I cant give specific details but i know for my NMAP tracker, my phone is listed in the known_devices.yaml which is where HA gets info for the frontend badge

yesimwilliam · February 2, 2018, 5:02am

Correct, but why is it that every time I restart HA it duplicated every entry in the known-devices file? So no matter if I turn scan new devices to false it still duplicated all entries… Drives me crazy. I’m so close to getting this to work.