Composite Device Tracker Platform

[Mariusthvdb (Marius)]

HI Phil, aware you have lost time for all things important, so just letting you know I notice a new (think…) issue in the logs, after having updated to HA 116:

Error while processing state changed for device_tracker.me
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/event.py", line 251, in _async_state_change_dispatcher
    hass.async_run_job(action, event)
  File "/usr/src/homeassistant/homeassistant/core.py", line 384, in async_run_job
    target(*args)
  File "/usr/src/homeassistant/homeassistant/helpers/event.py", line 844, in _refresh
    update = self._render_template_if_ready(track_template_, now, event)
  File "/usr/src/homeassistant/homeassistant/helpers/event.py", line 819, in _render_template_if_ready
    self._info[template] = template.async_render_to_info(track_template_.variables)
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 401, in async_render_to_info
    render_info._result = self.async_render(variables, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/template.py", line 332, in async_render
    return compiled.render(kwargs).strip()
  File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/local/lib/python3.8/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 4, in top-level template code
  File "/usr/local/lib/python3.8/site-packages/jinja2/sandbox.py", line 462, in call
    return __context.call(__obj, *args, **kwargs)
TypeError: tzinfo argument must be None or of a tzinfo subclass, not type 'Undefined'

and this is what’s in the Log viewer:

TemplateError('TypeError: tzinfo argument must be None or of a tzinfo subclass, not type 'Undefined'') while processing template 'Template("{% set state = states.device_tracker.me %} {% if state.attributes is not none and state.attributes.time_zone is not none and state.attributes.time_zone != 'unknown' %} {% set n = now() %} {{(n.astimezone(state.attributes.last_seen.tzinfo).utcoffset() - n.utcoffset()).total_seconds()/3600}} {% else %} undetermined {% endif %}")' for attribute '_state' in entity 'sensor.marijn_tz_offset'

have also tried to add a condition

        value_template: >
          {% set state = states.device_tracker.me %}
          {% if state.attributes is not none and
                state.attributes.time_zone is not none and
                state.attributes.time_zone != 'unknown'  and
               state.attributes.time_zone is defined %}
            {% set n = now() %}
            {{(n.astimezone(state.attributes.last_seen.tzinfo).utcoffset() -
               n.utcoffset()).total_seconds()/3600}}
          {% else %}
            undetermined
          {% endif %}

but none of these seem to prevent this from happening.

my config hasn’t changed in ages for this device_tracker, so maybe the HA update could have caused this?

seems the tracker is fine though (will have to see tomorrow when I move again ( ), so this could be a startup issue.

edit

no its definitely not a startup issue, happening at random (unexpected) moments. Maybe related to 116/117 (been running the beta’s for several days and still happening)

[pnbruckner (Phil)]

Did you ever get to the bottom of this?

Edit: Honestly, the only way I can see this error occurring is if something (other than the composite integration) has changed device_tracker.me's last_seen attribute to something other than a Python datetime object. Do you have anything else in your system that might change that attribute?

[pnbruckner (Phil)]

Released 2.1.3

Move version to manifest.json per new integration requirements. Also fix documentation link and add issue tracker link.

Released 2.2.0

Add support for TimezoneFinderL class from timezonefinder package.

[Mariusthvdb (Marius)]

Hi Phil,
ltns, hope you’re doing well.

tbh, I moved away from the composite tracker for some time now, and haven’t seen that error since.

I can easily reinstate it to have another check though.

my composite was made up of the life360 device_tracker, the wifi nmap tracker, ping, and owntracks. Tried to cover all bases

Don’t think any of these device_trackers has changed meanwhile so it would be interesting to see indeed.

I did at the time have an AD app running adding the attributes of geopy to the life360 trackers. I have indeed changed that, and created separate trackers for those, so the core Life360 tracker now is untouched. Could that have been it?

[pnbruckner (Phil)]

@Mariusthvdb

I did at the time have an AD app running adding the attributes of geopy to the life360 trackers. I have indeed changed that, and created separate trackers for those, so the core Life360 tracker now is untouched. Could that have been it?

Sure could have. Again, anything that changed the last_seen attribute to something other than a Python datetime object would have caused the error with your template; especially seeing as the “guards” you had in place were checking the time_zone attribute and not the last_seen attribute where the problem was.

[Mariusthvdb (Marius)]

Hi Phil,

given some sparkles in the new release thread on known_devices.yaml and the deprecated/phasing-out state of it, are you considering migrating the Composite CC in to a UI configured integration with config flow?
I would hate to lose the functionality of your integration if and when the HA main dev’s decide the use of known_devices.yaml is taken out…

[pnbruckner (Phil)]

As you may know, I don’t have nearly as much time to work on HA these days. So the basic answer to your question is, no, I do not plan to make any major changes to any of my custom integrations anytime soon. If this (and other) custom integrations break at some point I’ll probably stop updating my HA install. It pretty much does everything I want it to do, so I don’t have a lot of incentive to always stay on the latest release.

Some background…

A long time ago (just after I finished submitting the Life360 integration to become a standard integration) the device tracker component evolved to be “entity based.” In that transition, it maintained “legacy” support for most device tracker platforms (commonly referred to as “integrations” these days.) Only some of the device tracker integrations were updated to the new model at that time, and others have migrated since. The legacy support is what uses/maintains known_devices.yaml.

I’ve meant to migrate the Life360 integration to the new model, but that turned out to be more complicated than I expected, and I just didn’t have the time or energy to do it. So it is still considered a “legacy” device tracker integration and still uses known_devices.yaml. If/when this legacy support, and the use of known_devices.yaml, is completely removed, someone will have to upgrade the Life360 integration accordingly, or it, too, will be deprecated/removed.

If, someday, I actually spend time on upgrading the Life360 integration to be compliant with all the new rules and infrastructure, then I may spend time upgrading this custom integration as well based on what I would have learned doing the same with Life360. On the other hand, last I knew (and this may have changed), custom integrations were not allowed to use “config flow”, so???

Bottom line, no promises.

Of course, someone else might update the Life360 integration, and it’s always possible for anyone to fork this custom integration project and do the same.

[Mariusthvdb (Marius)]

thanks, and appreciated. This is very valuable information for the discussion some of us had in the other thread I linked.
Hope it wont get the far, and we will be able to use your CC for a long time still.

Btw, there are several Custom integrations that use the config flow, so yes, that is possible these days. Tahoma, Weatherbit, Eventsensor, to name a couple I use, probably others too.

[pnbruckner (Phil)]

Just an update/FYI, I’ve been able to spend some time on HA again recently. In fact, I have a PR open for converting the built-in Life360 integration to the new entity-based format. I still have to incorporate some significant review feedback / change requests, so that’s going fairly slowly.

Given my work on that, I have been considering converting this custom integration to the new format, possibly moving the configuration entirely to the UI. But that probably won’t happen very soon. It’s just on my list of possible to-dos at this point.

[pnbruckner (Phil)]

I received a request to allow non-GPS based “input” trackers / binary sensors to use all their states in determining the state of the composite tracker. E.g., change the state of the composite to not_home when a binary sensor changes to off. I have an initial implementation which does that selectively. I.e., the option to use all states of an input entity can be specified on an entity-by-entity basis. E.g.:

...
entity_id:
  - device_tracker.me
  - entity: binary_sensor.i_am_home
    all_states: true

If you’re interested in this functionality and would like to give it a try, see:

Add option to use all states of an input entity by pnbruckner · Pull Request #28 · pnbruckner/ha-composite-tracker (github.com)

[Ildar_Gabdullin]

@pnbruckner

possibly moving the configuration entirely to the UI

First, thank you for your composite platform very much!
Regarding moving to UI - I totally dislike this “move from yaml to UI” trend.
Surely, UI is more liked by some people - but I would like to have more control on my Composite sensors in part of “comment some code / uncomment some code” etc - which is not possible with UI.

Currently I may just comment my whole package for Composite, move it to another HA instance etc.
Doing similar when using UI is a pain.

OK, one of advantages of UI is “prevent user’s errors in a code”.
I guess that people should be responsible for their code. If I made an error - this is MY error.

[pnbruckner (Phil)]

I hear what you’re saying. I, too, prefer editing text (yaml) files vs changing via the UI. If/when I make any major modifications to this custom integration like that, I’ll consider maintaining the yaml-based config, even if it also allows total config via the UI. I believe both can exist side-by-side. And the good thing about creating a custom integration is I can make those decisions without having to get agreement from the (rest of) the core team (unless, of course, the infrastructure changes in such a way as to prevent the possibility.)

[Ildar_Gabdullin]

@pnbruckner

I’ll consider maintaining the yaml-based config, even if it also allows total config via the UI. I believe both can exist side-by-side.

I will pray for this)))
Good luck to you!

[Ildar_Gabdullin]

Continuation of this post.

Unfortunately, sometimes the Composite platform gave me not precise results.
Sorry for an off-topic.

Assume I have a few devices.
There are trackers for these devices:
– asuswrt, iphonedetect - these are “router” trackers, may only report home/not_home info;
– Life360, Traccar, Companion app - these are “gps trackers”.
All of “gps” trackers may report “battery_level” info.
All of “gps” trackers (except Traccar) may report “battery_charging” info.
Not every device has all 5 trackers - number of using trackers is ~2…5 dependently on a device.

Note that:
– Companion App in my practice gives an absolutely not reliable data for my 3 devices (data are usually outdated);
– Traccar is not working sometimes; last time it was not working since 28.06, was fixed in 2022.8;
– Life360 is usually NOT working in Russia at all, need to use OpenVPN - which is not working sometimes.

For every device I created a Composite tracker and listed all available trackers.
And a composite tracker was selected for a “person” entity as the only source.

What I noticed is:
1.Position reported by Composite may be not the actual last.
2.Battery_level reported by Composite may be not the actual last.
3.Battery_charging reported by Composite may be not the actual last.

What is the “actual value” - I can see when having data for all trackers displayed on one page.
In most cases the reported data are OK, but in some cases - not OK.

What I had to do is I created a template sensor for every device.
Called it “sensor.tracker_common_xxxxx” (xxxx - name of a device).
This sensor enumerates a list of available trackers.
For each info - position (zone + geo coords), battery_level, battery_charging - it does the following steps:
– find the latest info;
– store the time of this info (kind of “last_seen” but for a particular info);
– store a source for the latest info (like “life360”, “traccar”).

As a result - the “tracker_common” sensor provides 3 data packages (value, time, source) for position (zone+coords), battery_level, battery_charging.
And these results seem to be OK for me.

Unfortunately - I cannot use these data for a “person” entity as a source since it requires “device_tracker” source only.
So, now I am using 2 ways:
– composite tracker for a “person” entity;
– my “tracker_common” sensor to gather detailed (and more precise) info for position, battery data.

In automations for persons I am using “tracker_common” data (as well as for battery data for all other needs).
The “person” data (based on Composite) are for display only - and these data are displayed along with the “tracker_common” data to compare.
For example, here is a difference: the Composite says “home” but the device was actually outside (about 30 minutes):
– iPhoneDetect & ASUSWRT say “not_home” (which is true);
– Life360 & Traccar were not working.

I may share a code for my “tracker_common” sensor, also I would be happy if the Composite tracker provides more precise data.
And probably there may be errors in my sensor, will be great to fix them.

---

BTW, using the Composite (or a similar platform like my “tracker_common”) for a “person” is a great thing because of one more reason.
Assume we have 2 device trackers for a person:
– router-based;
– gps-based.
When the person is not home - the “person” entity uses the gps-based tracker and hence contains geo coodinates. So this person is displayed on a Map card.
When the person is home - in some cases (may be most) the “person” entity uses the router-based tracker and hence DOES NOT contain geo coordinates. So the person is NOT displayed on the Map - which I dislike.
When using Composite: even when the latest tracker is router-based, the geo coordinates are still present in the “device_tracker” entity - so they are present in the “person” entity - so the person is displayed on the Map.

[pnbruckner (Phil)]

Well, that is a LOT of information!

First, my basic response is, the composite tracker tries to do its best to use the information it’s getting from its multiple input entities. But if that input information if faulty, then it can’t “fix” that.

Second, the basic approach is to use the “latest” information from amongst the multiple input entities. When an input entity changes, the composite tries to determine if that entity has newer information than what it has seen up until that point. Some input entities have a last_seen attribute that categorically specifies when that data was last known to be valid. For others, the composite has to do the next best thing, which is to look at that entity’s last_updated state field. But that is just when the HA entity was last updated. That doesn’t mean the source of that data is that new. It could be some old value the server has cached from seconds or minutes ago. So, in this case the composite might have no choice but to use that data, even though it’s actually much older than previous data from an input entity that does have a last_seen attribute.

Now to comment on some of what you said…

@Ildar_Gabdullin

Unfortunately, sometimes the Composite platform gave me not precise results.

Garbage in, garbage out. There’s not much the composite tracker can do with bad or incomplete information from its input entities. If an input entity is updating with old data, but not properly timestamping that data, …

@Ildar_Gabdullin

Companion App in my practice gives an absolutely not reliable data for my 3 devices (data are usually outdated)

Well, then maybe you shouldn’t be using it as one of the input entities for the composite, unless it provides a valid timestamp attribute (such as last_seen) that the composite can use to ignore data older than what it has from other input entities.

@Ildar_Gabdullin

What I noticed is:
1.Position reported by Composite may be not the actual last.
2.Battery_level reported by Composite may be not the actual last.
3.Battery_charging reported by Composite may be not the actual last.

It’s going to be the last as reported by the input entities. Again, garbage in, garbage out.

@Ildar_Gabdullin

What I had to do is I created a template sensor for every device.

If you know some of these entities are providing bad data, and how to handle that, because you’ve spent the time to observer and understand their unique behavior, then yes, this would be a special case and would probably require special handling, such as template sensors and/or automations, to “filter” the data appropriately.

@Ildar_Gabdullin

Unfortunately - I cannot use these data for a “person” entity as a source since it requires “device_tracker” source only.

You could use the (still available) device_tracker.see service to create such an entity from your sensors (via an appropriate automation.) That is what I did long ago before I wrote the composite integration.

@Ildar_Gabdullin

For example, here is a difference: the Composite says “home” but the device was actually outside (about 30 minutes):
– iPhoneDetect & ASUSWRT say “not_home” (which is true);
– Life360 & Traccar were not working.

The composite has a recent new feature that allows you to tell it to use all the states of a particular input entity, such as the ones you mention. E.g.,:

device_tracker:
  - platform: composite
    name: me
    entity_id:
      - device_tracker.platform1_me
      - device_tracker.platform2_me
      - entity: device_tracker.router_my_device
        all_states: true
      - entity: binary_sensor.i_am_home
        all_states: true

Oh, wait! That’s still staged in a PR I haven’t merged yet. D’oh! I got so distracted by the Life360 work that I never got around to finishing that. I’ll try to do that soon.

@Ildar_Gabdullin

When the person is not home - the “person” entity uses the gps-based tracker and hence contains geo coodinates. So this person is displayed on a Map card.
When the person is home - in some cases (may be most) the “person” entity uses the router-based tracker and hence DOES NOT contain geo coordinates. So the person is NOT displayed on the Map - which I dislike.
When using Composite: even when the latest tracker is router-based, the geo coordinates are still present in the “device_tracker” entity - so they are present in the “person” entity - so the person is displayed on the Map.

It could be that the map treats a person entity different from a device_tracker entity, but last I understood, the map doesn’t show an entity when it is home, no matter what type it is, or whether or not it has GPS data. But I could be wrong. I don’t really use the person integration much (only in testing sometimes.)

[Ildar_Gabdullin]

@pnbruckner

First, my basic response is, the composite tracker tries to do its best to use the information it’s getting from its multiple input entities. But if that input information if faulty, then it can’t “fix” that.

Sure, but it should select the LATEST change for a position, battery_level & battery_charging - even if these data are wrong.

@pnbruckner

For others, the composite has to do the next best thing, which is to look at that entity’s last_updated state field. But that is just when the HA entity was last updated. That doesn’t mean the source of that data is that new. It could be some old value the server has cached from seconds or minutes ago. So, in this case the composite might have no choice but to use that data, even though it’s actually much older than previous data from an input entity that does have a last_seen attribute.

Absolutely right.
Here is what I managed to do:
– for Life360: use “device.tracker.last_seen”;
– for Traccar: use “device.tracker.last_seen” (but this attribute disappeared in 2022.8, have to use “device.tracker.last_updated”);
– for ASUSWRT: use “device.tracker.last_time_reachable”;
– for Companion App: “device.tracker.last_updated” - most unreliable data;
– for others: “device.tracker.last_updated”.

@pnbruckner

If an input entity is updating with old data, but not properly timestamping that data, …

Agree, but what I did is:
– place all data (last_updated/changed, last_seen, position, battery_data) on one page;
– compare these data.
So, even these “possibly garbage data” should be used by Composite if they are LATEST.
What I observed in SOME cases - Composite is not using LATEST data.
I repeat, it happened only in SOME cases - and mostly Composite provides LATEST data.

See again that example with iPhoneDetect tracker - is reported “not_home” (and ASUSWRT reported “not_home” too - and I was not really at home this time) - but Composite did not use this info.

@pnbruckner

Well, then maybe you shouldn’t be using it as one of the input entities for the composite, unless it provides a valid timestamp attribute (such as last_seen) that the composite can use to ignore data older than what it has from other input entities.

Correct, but unfortunately Companion tracker does not provide this info.
This device physically was not home for a month - but it is still “home”:

And “last_updated” is a time of HA reboot.

@pnbruckner

to “filter” the data appropriately.

But this “tracker_common” sensor does not actually “filter” data - it just selects the latest data.
W/o any further processing.

@pnbruckner

You could use the (still available) device_tracker.see service to create such an entity from your sensors

Was going to do that:
– create a dummy device_tracker (which is not going to be updated by any integration);
– use “see” service whenever my “tracker_common” server changes.
But - I simply do not know how to create this dummy device_tracker…
Each integration like Life360 etc updates it’s trackers.
Also, there is a “Demo” integration which can create device_tracker for “paulus”, “ann” etc.
And these trackers are moving randomly near my home (i.e. same city).
I haven’t tried to use the “see” service for these demo trackers 'cause I thought that these trackers are controlled by that “Demo” integration.

@pnbruckner

The composite has a recent new feature that allows you to tell it to use all the states of a particular input entity, such as the ones you mention.

Oh, I need to learn this feature with more attention!

@pnbruckner

It could be that the map treats a person entity different from a device_tracker entity, but last I understood, the map doesn’t show an entity when it is home, no matter what type it is, or whether or not it has GPS data

1. I managed to show “sensor.tracker_common_xxx” entities on the Map - since these sensors have geo coords.
2. Life360 trackers show “home” AND have geo coords - so they are displayed on the map.
3. Assume that Composite = Life360, ASUSWRT. And the “person” entity uses Composite as a source. Then the “person” is displayed on the Map if it is “home” - because Composite “passes” the “home” state and geo coords - so the geo coords are present in the “person” entity.
4. If “person” uses Life360 & ASUSWRT as sources - as I said earlier, when the latest source is ASUSWRT then the “person” entity does not have geo coords - and it is NOT displayed on the Map.

So, using Composite for “person” really helps if you want to have persons ALWAYS displayed on the Map.

[pnbruckner (Phil)]

@Ildar_Gabdullin

but it should select the LATEST change

As I said, it does, based on the data it is aware of (i.e., last_seen attribute or last_updated state field.)

@Ildar_Gabdullin

for ASUSWRT: use “device.tracker.last_time_reachable”

I was not aware of this attribute. I can certainly update the composite tracker to try to use this attribute, too, if the entity doesn’t have a last_seen attribute.

@Ildar_Gabdullin

Was going to do that:
– create a dummy device_tracker (which is not going to be updated by any integration);
– use “see” service whenever my “tracker_common” server changes.
But - I simply do not know how to create this dummy device_tracker…

You don’t explicitly create the entity. The first time you use the device_tracker.see service (with a particular dev_id value), the service will “create” it for you. I.e., it will create an entry in known_devices.yaml and write the state to the HA State Machine.

@Ildar_Gabdullin

@pnbruckner

The composite has a recent new feature that allows you to tell it to use all the states of a particular input entity, such as the ones you mention.

Oh, I need to learn this feature with more attention!

Again, I still need to find time to release this change.

[pnbruckner (Phil)]

Released 2.3.0

Add option to use all states of an input entity, which would typically be used with non-GPS device trackers or binary sensors.

[Tyzing (Ty)]

I’m a complete n00b so please forgive me. @pnbruckner thank you for all of your work with Life360, been using it for years. I too have an issue with “Unknown” which causes the automations to trigger in the middle of the night (my wife might kill me for this if I don’t fix it).

I installed Composite, but honestly I don’t know how to tell if it’s working lol. Here’s what I have in my config:

device_tracker:
  - platform: composite
    name: life360_wifename
    time_as: device_or_local
    entity_id:
      - device_tracker.life360_wifename
      - device_tracker.life360_myname

Did I set this up right? I’m just trying to eliminate the automations triggering due to unknown from life360. I only added my wife to the name because it seems she’s triggering that a lot more than I am. Not even sure if I did the “name” right but saw something about adding what you have after the “.” so gave that a try. Looked in the logs and didn’t see anything good or bad about Composite.

[pnbruckner (Phil)]

You need to create a composite entity for each of you. And you can’t use the same object ID as the “input” entity.

device_tracker:
  - platform: composite
    name: wifename
    time_as: device_or_local
    entity_id:
      - device_tracker.life360_wifename
  - platform: composite
    name: myname
    time_as: device_or_local
    entity_id:
      - device_tracker.life360_myname