Device trackers - conflicting More Info?

klogg · August 27, 2019, 12:33pm

Am I misunderstanding something fundamental or do these look wrong?

flamingm0e · August 27, 2019, 12:53pm

The bar shows multiple states. Are you actually home right now?

This is my son’s phone:

Here is mine:

pnbruckner · August 27, 2019, 1:27pm

That does seem weird. If you just left home I suppose the line graph might not be able to show the new state since it’s so small. If you click the “title bar” of the “more info” window (which should cause it to expand horizontally), do you still not see anything on the right end to show the Away state?

klogg · August 27, 2019, 1:47pm

@flamingm0e, @pnbruckner

This was my wife who was out, several miles away and had been so for a couple of hours.

pnbruckner · August 27, 2019, 2:05pm

Sounds to me like this entity is not getting recorded in the database anymore, or the database has become corrupted, or something along those lines.

klogg · August 27, 2019, 2:25pm

I think you might be on the right lines. But it’s even stranger…

Firstly, the Google device tracker is doing the same thing, and actually they are doing it for both me and my wife. however both Composite trackers behave as expected.

What is strange is that in my Logbook I only get entries for my Life360 and Google device trackers when they change to ‘Home’.

Any ideas what could be going on?

pnbruckner · August 27, 2019, 2:30pm

I’d query the database directly to see what state changes are getting recorded. Then go from there.

klogg · August 27, 2019, 3:14pm

I’m not sure how to do that…

pnbruckner · August 27, 2019, 3:16pm

What database do you use?

Alternatively, you can enable debug for homeassistant.core, and then each state change will get written to home-assistant.log. You’ll probably not want that on for long, especially if you have a “large” system, but it can help. I wrote a python script that can extract entity state changes from home-assistant.log if you’re interested.

klogg · August 27, 2019, 3:21pm

err… I use the database that comes with hassio.
I have recently installed the Influx and Grafana add-ons but am still finding my way around those.

I am interested in that script but I won’t be enabling debug today as I don’t expect any more state changes with the device trackers. I’ll try tomorrow.

flamingm0e · August 27, 2019, 3:22pm

FYI: it’s SQLITE

pnbruckner · August 27, 2019, 3:38pm

I use MariaDB, and that allows me to use mysql to query it while HA is running. I had no such luck when I used sqlite. But then I’m not all that familiar with databases.

Their states change all the time when you move just a little (i.e., the GPS coordinates change.) But you’re right, the state’s state doesn’t change if you’re not entering/leaving zones.

You can get the script here:

github.com

pnbruckner/homeassistant-config/blob/53fe1dff32a36a84a4efdbb87b72387d4ce5f091/tools/state_updates.py

#!/usr/bin/env python3

from collections import OrderedDict
import re
import sys


try:
    filename = sys.argv[1]
    if '.' not in sys.argv[2]:
        raise ValueError
except:
    print('Usage: python3 {} filename (entity_id [attribute...])...'.format(sys.argv[0]))
    sys.exit(1)

attrs = {}
entity_id = None
for arg in sys.argv[2:]:
    if '.' in arg:
        if entity_id is not None:

This file has been truncated. show original

The easiest way to use it is to copy it into the same directory as home-assistant.log. (Don’t forget to get a “raw” copy from github.) Then from the command line:

python3 state_updates.py home-assistant.log device_tracker.xxx

The first parameter is the name of the log file. Then you can list one or more entity_id’s (separated by spaces.) For each entity_id you can also follow with attribute name(s). But for this purpose, I’d just enter one device_tracker entity_id per invocation. It will list each “state” change (i.e., the state object’s state) along with the log’s timestamp and the state’s last_changed value. (You’ll note it also outputs the homeassistant.start event.)

klogg · August 27, 2019, 4:08pm

Thanks. I’m at the edge of my expertise here on several counts. Database amounts to IBM DB2 at least 20 to 25 years ago and MS Access 10 years ago; Linux command line and Python is entirely ‘learnt’ since using HA.

I’m not even sure if I can run a python script on the command line using hassio (in a VM on Proxmox) but I’ll see what I can do.

The frustrating thing is that HA seems to work ok so it is all the more strange that this is happening?

pnbruckner · August 27, 2019, 4:13pm

I hear you about hassio. That’s why I’ve never even considered using it. On the other hand I guess you don’t miss what you don’t know.

The script (if you can run it) will tell you when the entities change state and to what. It will confirm what the history should be showing. But it can’t really tell you why the history isn’t showing what it should. For that you do really need to query the database. Of course, that’s what the history graph in the more info window is supposed to be doing.

Are you sure you haven’t excluded these entities from the recorder?

klogg · August 27, 2019, 4:31pm

That’s the first thing I checked because I have recently made changes there because my snapshots were getting huge, but my recorder is as follows:

recorder:
  purge_keep_days: 3
  purge_interval: 1

  exclude:
    domains:
      - camera
      - counter
      - group
      - history_graph
      - proximity
      - updater
      - weather
      - weblink
      - zone

    entities:

Followed by some specific Automations, Binary Sensors, Media Players, Scripts and Sensors

pnbruckner · August 27, 2019, 4:37pm

Ok, cool.

Bottom line (for me) is the symptoms seem to indicate there is some history for the device_trackers (it’s finding something old), but new state changes aren’t getting saved. Until and unless you can find a way to see what’s really in the db, I think it’s going to be near impossible, if not impossible, to get to the bottom of this.

I suppose it could also simply be that the database somehow got corrupted. Is there anything in home-assistant.log that might indicate issues with the db? I know it’s not optimal, but if you delete the db does it happen again?

klogg · August 27, 2019, 4:49pm

Hmm, I was considering that but really don’t want to. I will as a last resort though.

What about removing the integrations and re-instating them? Is that an option, by which I mean would HA internally forget it ever knew they existed and recreate them or will it just carry on where it left off?

Actually, re-reading this:

I am not sure if that is right. It is always saving the state when it changes to ‘home’ but not when it changes to ‘away’.

Tomorrow I will be able to see if it saves the state when it changes to another zone. (By which I mean I will be able to se in the Logbook).

pnbruckner · August 27, 2019, 4:55pm

I can’t imagine that will help at all.

Um, if it only “saves” when it changes to home, how could you possibly see that? Honestly, I’m not sure where the Logbook gets its data, so I couldn’t say how what you see there relates to the database (and ultimately the history shown on the history page, or in an entity’s more info window.)

If I were you I’d concentrate on trying to find a way to see what’s in the database. Is there some sort of hassio add-on that provides database querying capability? Or maybe you can stop HA, copy the database file to another system, and view it there with some utility???

klogg · August 27, 2019, 5:03pm

Yeah, that’s the only way I know it “saves” the state - because it shows it happening in the Logbook. so, no, I suppose you’re right, I don’t know if it is saving!

I’ll look into it - I think there are hassio DB add-ons…

klogg · August 28, 2019, 12:14pm

I don’t know where this leaves us me, but there is a (rather good) SQLLite add-on for hassio and I can confirm that all the Life360 states are being written to the database. That is, home, not_home and zone names.

So why does the more-info display wrongly and the Logbook not show all the data?

Also (and this might be a separate issue, I’m happy to start a new post) Life360 looks to me like it is reporting some strange results.

For example it changes from a zone where it has been for several hours to ’not_home for less than two seconds, then back to the zone; before flipping again 3 minutes later for less than a second before returning back to the zone.

The interesting bit (to me) is that these not_home states have a "source_type": null.

I can see this behaviour other times too, including when it is home and has been for several hours but then jumps momentarily to not_home with a "source_type": null

When it really is home it seems to have the correct "source_type": "gps".

I know this is all getting a bit esoteric but I’m grateful for your thoughts. (I can of course provide actual data if it helps or is interesting)