Composite Device Tracker Platform

[pnbruckner (Phil)]

Just an update/FYI, I’ve been able to spend some time on HA again recently. In fact, I have a PR open for converting the built-in Life360 integration to the new entity-based format. I still have to incorporate some significant review feedback / change requests, so that’s going fairly slowly.

Given my work on that, I have been considering converting this custom integration to the new format, possibly moving the configuration entirely to the UI. But that probably won’t happen very soon. It’s just on my list of possible to-dos at this point.

[pnbruckner (Phil)]

I received a request to allow non-GPS based “input” trackers / binary sensors to use all their states in determining the state of the composite tracker. E.g., change the state of the composite to not_home when a binary sensor changes to off. I have an initial implementation which does that selectively. I.e., the option to use all states of an input entity can be specified on an entity-by-entity basis. E.g.:

...
entity_id:
  - device_tracker.me
  - entity: binary_sensor.i_am_home
    all_states: true

If you’re interested in this functionality and would like to give it a try, see:

Add option to use all states of an input entity by pnbruckner · Pull Request #28 · pnbruckner/ha-composite-tracker (github.com)

[Ildar_Gabdullin]

@pnbruckner

possibly moving the configuration entirely to the UI

First, thank you for your composite platform very much!
Regarding moving to UI - I totally dislike this “move from yaml to UI” trend.
Surely, UI is more liked by some people - but I would like to have more control on my Composite sensors in part of “comment some code / uncomment some code” etc - which is not possible with UI.

Currently I may just comment my whole package for Composite, move it to another HA instance etc.
Doing similar when using UI is a pain.

OK, one of advantages of UI is “prevent user’s errors in a code”.
I guess that people should be responsible for their code. If I made an error - this is MY error.

[pnbruckner (Phil)]

I hear what you’re saying. I, too, prefer editing text (yaml) files vs changing via the UI. If/when I make any major modifications to this custom integration like that, I’ll consider maintaining the yaml-based config, even if it also allows total config via the UI. I believe both can exist side-by-side. And the good thing about creating a custom integration is I can make those decisions without having to get agreement from the (rest of) the core team (unless, of course, the infrastructure changes in such a way as to prevent the possibility.)

[Ildar_Gabdullin]

@pnbruckner

I’ll consider maintaining the yaml-based config, even if it also allows total config via the UI. I believe both can exist side-by-side.

I will pray for this)))
Good luck to you!

[Ildar_Gabdullin]

Continuation of this post.

Unfortunately, sometimes the Composite platform gave me not precise results.
Sorry for an off-topic.

Assume I have a few devices.
There are trackers for these devices:
– asuswrt, iphonedetect - these are “router” trackers, may only report home/not_home info;
– Life360, Traccar, Companion app - these are “gps trackers”.
All of “gps” trackers may report “battery_level” info.
All of “gps” trackers (except Traccar) may report “battery_charging” info.
Not every device has all 5 trackers - number of using trackers is ~2…5 dependently on a device.

Note that:
– Companion App in my practice gives an absolutely not reliable data for my 3 devices (data are usually outdated);
– Traccar is not working sometimes; last time it was not working since 28.06, was fixed in 2022.8;
– Life360 is usually NOT working in Russia at all, need to use OpenVPN - which is not working sometimes.

For every device I created a Composite tracker and listed all available trackers.
And a composite tracker was selected for a “person” entity as the only source.

What I noticed is:
1.Position reported by Composite may be not the actual last.
2.Battery_level reported by Composite may be not the actual last.
3.Battery_charging reported by Composite may be not the actual last.

What is the “actual value” - I can see when having data for all trackers displayed on one page.
In most cases the reported data are OK, but in some cases - not OK.

What I had to do is I created a template sensor for every device.
Called it “sensor.tracker_common_xxxxx” (xxxx - name of a device).
This sensor enumerates a list of available trackers.
For each info - position (zone + geo coords), battery_level, battery_charging - it does the following steps:
– find the latest info;
– store the time of this info (kind of “last_seen” but for a particular info);
– store a source for the latest info (like “life360”, “traccar”).

As a result - the “tracker_common” sensor provides 3 data packages (value, time, source) for position (zone+coords), battery_level, battery_charging.
And these results seem to be OK for me.

Unfortunately - I cannot use these data for a “person” entity as a source since it requires “device_tracker” source only.
So, now I am using 2 ways:
– composite tracker for a “person” entity;
– my “tracker_common” sensor to gather detailed (and more precise) info for position, battery data.

In automations for persons I am using “tracker_common” data (as well as for battery data for all other needs).
The “person” data (based on Composite) are for display only - and these data are displayed along with the “tracker_common” data to compare.
For example, here is a difference: the Composite says “home” but the device was actually outside (about 30 minutes):
– iPhoneDetect & ASUSWRT say “not_home” (which is true);
– Life360 & Traccar were not working.

I may share a code for my “tracker_common” sensor, also I would be happy if the Composite tracker provides more precise data.
And probably there may be errors in my sensor, will be great to fix them.

---

BTW, using the Composite (or a similar platform like my “tracker_common”) for a “person” is a great thing because of one more reason.
Assume we have 2 device trackers for a person:
– router-based;
– gps-based.
When the person is not home - the “person” entity uses the gps-based tracker and hence contains geo coodinates. So this person is displayed on a Map card.
When the person is home - in some cases (may be most) the “person” entity uses the router-based tracker and hence DOES NOT contain geo coordinates. So the person is NOT displayed on the Map - which I dislike.
When using Composite: even when the latest tracker is router-based, the geo coordinates are still present in the “device_tracker” entity - so they are present in the “person” entity - so the person is displayed on the Map.

[pnbruckner (Phil)]

Well, that is a LOT of information!

First, my basic response is, the composite tracker tries to do its best to use the information it’s getting from its multiple input entities. But if that input information if faulty, then it can’t “fix” that.

Second, the basic approach is to use the “latest” information from amongst the multiple input entities. When an input entity changes, the composite tries to determine if that entity has newer information than what it has seen up until that point. Some input entities have a last_seen attribute that categorically specifies when that data was last known to be valid. For others, the composite has to do the next best thing, which is to look at that entity’s last_updated state field. But that is just when the HA entity was last updated. That doesn’t mean the source of that data is that new. It could be some old value the server has cached from seconds or minutes ago. So, in this case the composite might have no choice but to use that data, even though it’s actually much older than previous data from an input entity that does have a last_seen attribute.

Now to comment on some of what you said…

@Ildar_Gabdullin

Unfortunately, sometimes the Composite platform gave me not precise results.

Garbage in, garbage out. There’s not much the composite tracker can do with bad or incomplete information from its input entities. If an input entity is updating with old data, but not properly timestamping that data, …

@Ildar_Gabdullin

Companion App in my practice gives an absolutely not reliable data for my 3 devices (data are usually outdated)

Well, then maybe you shouldn’t be using it as one of the input entities for the composite, unless it provides a valid timestamp attribute (such as last_seen) that the composite can use to ignore data older than what it has from other input entities.

@Ildar_Gabdullin

What I noticed is:
1.Position reported by Composite may be not the actual last.
2.Battery_level reported by Composite may be not the actual last.
3.Battery_charging reported by Composite may be not the actual last.

It’s going to be the last as reported by the input entities. Again, garbage in, garbage out.

@Ildar_Gabdullin

What I had to do is I created a template sensor for every device.

If you know some of these entities are providing bad data, and how to handle that, because you’ve spent the time to observer and understand their unique behavior, then yes, this would be a special case and would probably require special handling, such as template sensors and/or automations, to “filter” the data appropriately.

@Ildar_Gabdullin

Unfortunately - I cannot use these data for a “person” entity as a source since it requires “device_tracker” source only.

You could use the (still available) device_tracker.see service to create such an entity from your sensors (via an appropriate automation.) That is what I did long ago before I wrote the composite integration.

@Ildar_Gabdullin

For example, here is a difference: the Composite says “home” but the device was actually outside (about 30 minutes):
– iPhoneDetect & ASUSWRT say “not_home” (which is true);
– Life360 & Traccar were not working.

The composite has a recent new feature that allows you to tell it to use all the states of a particular input entity, such as the ones you mention. E.g.,:

device_tracker:
  - platform: composite
    name: me
    entity_id:
      - device_tracker.platform1_me
      - device_tracker.platform2_me
      - entity: device_tracker.router_my_device
        all_states: true
      - entity: binary_sensor.i_am_home
        all_states: true

Oh, wait! That’s still staged in a PR I haven’t merged yet. D’oh! I got so distracted by the Life360 work that I never got around to finishing that. I’ll try to do that soon.

@Ildar_Gabdullin

When the person is not home - the “person” entity uses the gps-based tracker and hence contains geo coodinates. So this person is displayed on a Map card.
When the person is home - in some cases (may be most) the “person” entity uses the router-based tracker and hence DOES NOT contain geo coordinates. So the person is NOT displayed on the Map - which I dislike.
When using Composite: even when the latest tracker is router-based, the geo coordinates are still present in the “device_tracker” entity - so they are present in the “person” entity - so the person is displayed on the Map.

It could be that the map treats a person entity different from a device_tracker entity, but last I understood, the map doesn’t show an entity when it is home, no matter what type it is, or whether or not it has GPS data. But I could be wrong. I don’t really use the person integration much (only in testing sometimes.)

[Ildar_Gabdullin]

@pnbruckner

First, my basic response is, the composite tracker tries to do its best to use the information it’s getting from its multiple input entities. But if that input information if faulty, then it can’t “fix” that.

Sure, but it should select the LATEST change for a position, battery_level & battery_charging - even if these data are wrong.

@pnbruckner

For others, the composite has to do the next best thing, which is to look at that entity’s last_updated state field. But that is just when the HA entity was last updated. That doesn’t mean the source of that data is that new. It could be some old value the server has cached from seconds or minutes ago. So, in this case the composite might have no choice but to use that data, even though it’s actually much older than previous data from an input entity that does have a last_seen attribute.

Absolutely right.
Here is what I managed to do:
– for Life360: use “device.tracker.last_seen”;
– for Traccar: use “device.tracker.last_seen” (but this attribute disappeared in 2022.8, have to use “device.tracker.last_updated”);
– for ASUSWRT: use “device.tracker.last_time_reachable”;
– for Companion App: “device.tracker.last_updated” - most unreliable data;
– for others: “device.tracker.last_updated”.

@pnbruckner

If an input entity is updating with old data, but not properly timestamping that data, …

Agree, but what I did is:
– place all data (last_updated/changed, last_seen, position, battery_data) on one page;
– compare these data.
So, even these “possibly garbage data” should be used by Composite if they are LATEST.
What I observed in SOME cases - Composite is not using LATEST data.
I repeat, it happened only in SOME cases - and mostly Composite provides LATEST data.

See again that example with iPhoneDetect tracker - is reported “not_home” (and ASUSWRT reported “not_home” too - and I was not really at home this time) - but Composite did not use this info.

@pnbruckner

Well, then maybe you shouldn’t be using it as one of the input entities for the composite, unless it provides a valid timestamp attribute (such as last_seen) that the composite can use to ignore data older than what it has from other input entities.

Correct, but unfortunately Companion tracker does not provide this info.
This device physically was not home for a month - but it is still “home”:

And “last_updated” is a time of HA reboot.

@pnbruckner

to “filter” the data appropriately.

But this “tracker_common” sensor does not actually “filter” data - it just selects the latest data.
W/o any further processing.

@pnbruckner

You could use the (still available) device_tracker.see service to create such an entity from your sensors

Was going to do that:
– create a dummy device_tracker (which is not going to be updated by any integration);
– use “see” service whenever my “tracker_common” server changes.
But - I simply do not know how to create this dummy device_tracker…
Each integration like Life360 etc updates it’s trackers.
Also, there is a “Demo” integration which can create device_tracker for “paulus”, “ann” etc.
And these trackers are moving randomly near my home (i.e. same city).
I haven’t tried to use the “see” service for these demo trackers 'cause I thought that these trackers are controlled by that “Demo” integration.

@pnbruckner

The composite has a recent new feature that allows you to tell it to use all the states of a particular input entity, such as the ones you mention.

Oh, I need to learn this feature with more attention!

@pnbruckner

It could be that the map treats a person entity different from a device_tracker entity, but last I understood, the map doesn’t show an entity when it is home, no matter what type it is, or whether or not it has GPS data

1. I managed to show “sensor.tracker_common_xxx” entities on the Map - since these sensors have geo coords.
2. Life360 trackers show “home” AND have geo coords - so they are displayed on the map.
3. Assume that Composite = Life360, ASUSWRT. And the “person” entity uses Composite as a source. Then the “person” is displayed on the Map if it is “home” - because Composite “passes” the “home” state and geo coords - so the geo coords are present in the “person” entity.
4. If “person” uses Life360 & ASUSWRT as sources - as I said earlier, when the latest source is ASUSWRT then the “person” entity does not have geo coords - and it is NOT displayed on the Map.

So, using Composite for “person” really helps if you want to have persons ALWAYS displayed on the Map.

[pnbruckner (Phil)]

@Ildar_Gabdullin

but it should select the LATEST change

As I said, it does, based on the data it is aware of (i.e., last_seen attribute or last_updated state field.)

@Ildar_Gabdullin

for ASUSWRT: use “device.tracker.last_time_reachable”

I was not aware of this attribute. I can certainly update the composite tracker to try to use this attribute, too, if the entity doesn’t have a last_seen attribute.

@Ildar_Gabdullin

Was going to do that:
– create a dummy device_tracker (which is not going to be updated by any integration);
– use “see” service whenever my “tracker_common” server changes.
But - I simply do not know how to create this dummy device_tracker…

You don’t explicitly create the entity. The first time you use the device_tracker.see service (with a particular dev_id value), the service will “create” it for you. I.e., it will create an entry in known_devices.yaml and write the state to the HA State Machine.

@Ildar_Gabdullin

@pnbruckner

The composite has a recent new feature that allows you to tell it to use all the states of a particular input entity, such as the ones you mention.

Oh, I need to learn this feature with more attention!

Again, I still need to find time to release this change.

[pnbruckner (Phil)]

Released 2.3.0

Add option to use all states of an input entity, which would typically be used with non-GPS device trackers or binary sensors.

[Tyzing (Ty)]

I’m a complete n00b so please forgive me. @pnbruckner thank you for all of your work with Life360, been using it for years. I too have an issue with “Unknown” which causes the automations to trigger in the middle of the night (my wife might kill me for this if I don’t fix it).

I installed Composite, but honestly I don’t know how to tell if it’s working lol. Here’s what I have in my config:

device_tracker:
  - platform: composite
    name: life360_wifename
    time_as: device_or_local
    entity_id:
      - device_tracker.life360_wifename
      - device_tracker.life360_myname

Did I set this up right? I’m just trying to eliminate the automations triggering due to unknown from life360. I only added my wife to the name because it seems she’s triggering that a lot more than I am. Not even sure if I did the “name” right but saw something about adding what you have after the “.” so gave that a try. Looked in the logs and didn’t see anything good or bad about Composite.

[pnbruckner (Phil)]

You need to create a composite entity for each of you. And you can’t use the same object ID as the “input” entity.

device_tracker:
  - platform: composite
    name: wifename
    time_as: device_or_local
    entity_id:
      - device_tracker.life360_wifename
  - platform: composite
    name: myname
    time_as: device_or_local
    entity_id:
      - device_tracker.life360_myname

[pnbruckner (Phil)]

I’m working on converting this custom integration from a “legacy” device tracker to the newer entity-based implementation, similar to what I recently did with the Life360 integration.

The main reasons to do this are:

1. Legacy support in the device tracker component could go away at any time.
2. Move away from the very old known_devices.yaml file to the newer method of storing configuration and entity settings in the .storage folder.
3. Use a unique ID for the entities so that they can be managed via the Entities page like other entities.

For now, the configuration will still come from YAML files. It will be automatically imported into the .storage folder, and any updates to YAML will automatically update .storage as well. At some point in the future, I’ll probably add the ability to create the entities via the Integrations page. (For now, the imported ones will show up there.) I’ll probably never remove support for YAML configuration.

This time, unlike for what happened with Life360, I’m trying to make the changeover a bit smoother. (I’m a bit freer to do whatever I want!) E.g., it will continue working as before after upgrading (i.e., as a legacy tracker that uses known_devices.yaml), but will issue a deprecation warning about how to modify the YAML configuration to convert to the new implementation. Basically, move the following:

device_tracker:
  - platform: composite
    name: me
    time_as: device_or_local
    entity_id:
      - device_tracker.platform1_me
      - device_tracker.platform2_me
      - binary_sensor.i_am_home

to:

composite:
  trackers:
  - name: me
    time_as: device_or_local
    entity_id:
      - device_tracker.platform1_me
      - device_tracker.platform2_me
      - binary_sensor.i_am_home

At some point I’ll remove support for the old legacy configuration.

If anyone would like to give it a try, you can find the “work in progress” here:

pnbruckner/ha-composite-tracker at entity-based (github.com)

I haven’t updated the readme yet, though.

I’ve done some testing on 2022.10 w/ Python 3.9, but mostly I’ve been testing on 2021.3.3 w/ Python 3.8, since that’s what I’m still using on my “production” system. (Don’t ask! ) I want to use it on a daily basis to hopefully flush out any bugs I overlooked.

If anyone does give it a try, I’d really appreciate it! Let me know how it works for you, and what versions you’re using. THANKS!

[Ildar_Gabdullin]

@pnbruckner

I’ll probably never remove support for YAML configuration.

That will be really great!

@pnbruckner

move the following:

Sorry for a silly question, I just want to be sure - shall we do it NOW with the CURRENT version? Or is this related to the “work in progress” version you mentioned?

[pnbruckner (Phil)]

Wait for the new version.

[pnbruckner (Phil)]

Do you use HACS? I could try publishing a beta release. I think I know how to do that in a way that satisfies HACS, but I haven’t exactly tried it yet.

[pnbruckner (Phil)]

Released 2.4.0b0

This is a prerelease. It adds the newer entity-based implementation, while maintaining (deprecated) support for the original (now called “legacy”) implementation.

I’d be interested to hear how this pre-release works with HACS. And, of course, any feedback, good or bad, if anyone tries it.

[Ildar_Gabdullin]

Great, gonna install the beta release by HACS and test, thank you !

[finity]

For some reason I’m not finding it in HACS. Nether the existing or the dev version

[Edwin_D]

This integration is not needed any longer as far as I can tell, because you can add multiple device trackers to a person in HA. It pretty much does what this integration intends to: determine the location of a person using multiple sources.