Enhanced Google Maps Integration

Share your Projects! Custom Integrations
49

I am currently using two accounts with the integration – my personal account (which sees all the people who have shared their location with my account), and a “HA only” account (which only has my personal account shared with it.)

I compared the cookies that those two accounts have vs the cookies yours has. Before I talk about the similarities and differences, I’ll first say that I also see exactly the following for both of my accounts:

Saving cookies, changes: updated: __Secure-1PSIDCC, SIDCC, __Secure-3PSIDCC

All three accounts have these cookies in common:

APISID, HSID, NID, SAPISID, SEARCH_SAMESITE, SID, SIDCC, SSID, __Secure-1PAPISID, __Secure-1PSID, __Secure-1PSIDCC, __Secure-3PAPISID, __Secure-3PSID, __Secure-3PSIDCC

FWIW, __Secure-1PSID & __Secure-3PSID have been considered the important ones.

My two accounts each have a few cookies the other doesn’t. E.g., in addition to the common set, one has:

1P_JAR

whereas the other has:

__Secure-1PSIDTS, __Secure-3PSIDTS

Then I checked which cookies yours has that neither of mine has, which are:

AEC, AID, ANID, SNID, TAID, _ga, _ga_B7W0ZKZYDK, _gcl_au

EDIT: I should also say, my accounts do not have any cookies that yours doesn’t.

I’m not exactly sure what all this means, but it’s at least interesting.

Did you use, and exactly follow, one of the procedures my integration provides for obtaining the cookies file, or did you use some other method? And if the latter, can you explain what you did?

50

Thank you for this custom integration. The UI helped me configure my account. The problem I am facing is the dummy account I set up (lets call it Account B) shows up with a state UNKNOWN. I followed the steps outlined in the HA version: I created Account B and shared my personal account (lets call it Account A) location. I was under the impression Account A would be tracked via the integration but it doesn’t seem to be the case. Its worth noting, I do not have location tracking set up for Account B. Any idea what I am doing wrong? Thanks!

Edit: while I was successful at first, it appears the integration now keeps failing (after about 20 minutes or so), requiring reconfiguration via Cookies file. Not sure if this is related to my original issue or separate entirely.

51

So in your configuration, you should disable the “Account Tracker Entity” so that it does not try to create a device_tracker entity for Account B since it is not associated with a device that reports location. It should still, however, create a device_tracker entity for Account A, which has its location shared with Account B. (As you thought, it should get A’s location via B.) Do you see a tracker entity for Account A?

If you don’t see an entity for Account A, check in Google Maps for Account A and make sure it is sharing its location with Account B.

That sounds like the experience some people have. (E.g., see the posts just above with @dormani.) No-one quite seems to know why this happens. It certainly doesn’t seem to happen for most people, at least, not that I’m aware of.

I’ll ask you the same questions.

  • What version of my google_maps custom integration are you using?
  • Do you have debug enabled? If not, could you please enable debug?
  • Can you share the exact ERROR message(s) from the log?
  • Can you share the DEBUG messages about cookies?
  • Can you share (privately if necessary), the locationsharinglib DEBUG message that immediately preceded the ERROR?
  • If you no longer have access to those messages, can you start over (i.e., grab a new cookies file) and capture them?
  • What part of the world are you in?
  • Exactly how did you create Account B? Did you log out of Account B, and close the browser you used, after creating it and before grabbing the cookies file?
  • Exactly how did you grab the cookies file? Did you follow closely one of the procedures my integration describes?
53

Thanks! I didn’t see Account B at first but realized that my initial location share link was only for an hour. After updating it now appears.

My second problem appears to have been sorted. I followed your steps closely for cookies download and missed a few key steps (e.g. using private browser) on my first few passes.

1 Like
54

Released 1.2.0b0 (beta)

Replace locationsharinglib, reorg code & other misc improvements

  • Create replacement for locationsharinglib which more directly implements capabilities needed/used by integration.
  • Migrate configuration entry from version 1 to 2.
  • Add note to README.md about purging history of legacy trackers.
  • Remove support for initial beta versions when it comes to how entities are restored.
  • Write cookies & parsed data to log when invalid session indicated.
  • When loading new cookies file, store in newly named file in .storage/google_maps instead of overwriting existing file. This makes code much cleaner. Existing file will be deleted when config is reloaded.
  • Reorganize code to simplify, creating new helpers & coordinator modules.

I’ve released this as a beta because it was quite a substantial change. Nevertheless, I’d really appreciate some usage & feedback so I can feel more confident in fully releasing the changes.

A quick word of warning: If you do upgrade, the configuration entries will be updated from version 1 to 2. This means you will not be able to “downgrade” back to 1.1.1 or earlier without first deleting the entries, and then adding them back after downgrading. Hopefully this should not be necessary. :smile:

1 Like
55

If you’ve ever experienced the integration throwing the ERROR about invalid cookies or an invalid session, or with the built-in integration where it seems to just stop updating (until you restart, at which time you more than likely get the invalid cookies error), then this might interest you…

There still seems to be a bit of a mystery why this happens to some but not to others, which is not totally surprising because, like many HA integrations, it’s using an undocumented & unsupported API. But given some recent data, I might have an idea why, or at least when, it happens.

The basic approach of this integration is to re-use a “session” from a browser that has logged into Google by dumping its cookies. The HA integration then uses those cookies, effectively pretending to be the original browser session.

It seems for many (including myself), it doesn’t really matter which Google account or accounts is/are used with the HA integration, how it was, or they were, created, and exactly how the cookies (in one or more files) were obtained. It just seems to work (for at least a year or two before those cookies eventually expire, or with the recent changes to save updated cookies, maybe forever.)

For others, either the cookies never work, or more likely, they work for a while (maybe 20 minutes, maybe a few hours, maybe a few days), but eventually they just stop working.

The problem seems to start, at least for some, when the original browser session used to obtain the cookies continues to interact with Google, either relatively immediately, or maybe at a later time/date. The Google server will send updated cookies to both “copies” of the session, and those sessions will resend them back to the Google server with each requested update.

This doesn’t seem to be a problem for many but is for some.

I think the underlying problem is that the Google server sees two different responses (one from the browser, and the other from HA) to what it sees as, or should be, a unique login session. Maybe it sees this as some sort of “man in the middle” attack, and maybe it has different rules in different parts of the world. Whatever the reason, it seems to respond by “invalidating” the session, at least for the HA integration.

With this in mind, I wrote the procedures for obtaining the cookies purposely to use a “private” browser window. Private browser windows (I think) typically don’t save cookies after the window is closed. Therefore, the browser shouldn’t continue to attempt to use that same login session (which will now be used by HA), even if the browser is reopened afterwards.

To expand on those procedures, if you’re experiencing this problem, my recommendation is to make sure ALL sessions for the account or accounts used with HA are closed / signed-out, and all associated browsers are closed, BEFORE following the procedure for grabbing the cookies file in a private window. This probably means the only practical “account strategy” (for people experiencing this problem) is the one I called “Alt acct only”. Also, it may or may not be important to never relog into that account in a browser or other device (except if the cookies ever expire, I guess.)

Before continuing, you should probably delete the current Google Maps integration entries in HA.

To make sure all sessions are closed / signed-out, do the following:

  1. Go to Google Account in a private browser window.
  2. Log into the account used by HA.
  3. Go to Security → “Manage all device” (under “Your devices”). All currently signed in sessions (including the one you just started) should be listed. Click on, and sign-out of, all of those sessions. You won’t, of course, be able to sign-out of the current session (probably listed at the top) this way.
  4. Now sign-out of the current session (via the icon in the top-right part of the window), and close this private browser window.)

Now go ahead and grab a new cookies file and add the Google Maps integration entry to HA.

I’d appreciate any thoughts on this theory, and if anyone who has been experiencing the problem tries the above, please let me know how it works for you, good or bad. I could, of course, be totally wrong here, but :crossed_fingers:

4 Likes
56

Released 1.2.0b1

Fixes a bug that causes entire server update to fail if one or more people are not currently sharing location data, or that location data is missing for some other reason.

Previously this just caused the associated tracker entity to not update, but all the remaining ones would update. The fix makes it work the way it did before in this case, including writing a DEBUG message to the log when it happens.

57

Released 1.2.0 per above

58

Released 1.3.0

Attributes address & nickname were already not recorded in the database. entity_picture has a relatively large value and will almost never change, so add it to the list of attributes to not record.

59

I have just installed this integration.
I first used my google email address, did the cookie thing, and finished the process.
I went to the integrations page and my account was set up/

I then tried the same exact process for my wife’s phone using her gmail address.
In the integartion hers shows as unknown.

Does something need to be turned on from her phone. Or have I messed up a step in the process.

Thank you

60

Can you elaborate by what you mean by “in the integration”?

Normally, most people will try to use only one Google account with this integration.

E.g., you might use just your own Google account, and then have your wife share her location with you (which is done within the Google Maps app, not HA.) In this scenario, a device_tracker entity will be created for your wife. And, if you checked the box to create a tracker for the “account holder”, then an entity will be created for you as well.

What is often better, though, is to create a new Google account that you add to this integration. Then both you and your wife would share your locations with that account. In this scenario tracker entities will be created for both of you (and you would not check the box to create a tracker for the “account holder.”)

61

Thank you for the quick response.
If I create another gmail account that both my wife and I can share, I will use that new gmail account in the integration using it’s cookies.txt file.
Then I need to share the new gmail account.
Is that basically it?

Thank you

62

No, sorry, I don’t think you are following me.

You create a new account. You get the cookies.txt file for that account. You add that account to the HA integration.

Then you go into your Google Maps app on your phone and share your location with that new account.

Then you wife does the same.

You and your wife are not sharing the new account. You are sharing your locations from your existing accounts with the new account.

63

Got. Thats what I figured out eventually. Works fine.
Now how do I incorporate it into HA to get data from it that I can put on my dashboard.
Thanks

64

Once you get the integration working, you should see device_tracker entities, one for you and one for your wife.

You should be able to see them by going to Settings → Devices & services → Google Maps. Under “Integration entries”, you should see the account you added, and under that it should have a link named “2 entities”. Click on that and it will list the device_tracker entities. You can click on them to see their “more info” window.

If you’re asking how to use device_tracker entities in HA, that’s kind of a HA 101 question. But basically, you’ll probably want to create Person entities for yourself and your wife. Go to Settings → People. There’s probably already an entry there for you. Click it. On the bottom of the window that pops up, you can select your tracker entity. Then click Update.

Now click the “+ ADD PERSON” button at the bottom-right of the page and create a person entity for your wife, selecting her tracker at the bottom.

65

Found it.
I needed to clear cache and then reboot before it saw the info.
Been using ha almost from the begining but there slight changes with every thing you do.

Again got it all working.
Thanks to you and the community for all the help.

66

Ok. Yeah, not sure why the integration page doesn’t always update when it should, but I’ve always found a simple refresh of the browser window solves the problem. You shouldn’t have had to clear the cache and reboot. But you got it working, so that’s what’s important.

Glad you got it working! Let me know if you need any more help with this integration or if you have any other questions about it.

2 Likes
67

Hi,

Can’t we use both methods to get a tracker? I use your integration, but I also left the normal definition to have a tracker in the configuration.yaml file. I see the following message in my log.

IMG_8713
IMG_87132227×526 132 KB

68

I left in the legacy implementation so that simply installing this integration would not break a system. You’ll notice that is a warning, not an error.

However, until you change to the new configuration method, you’ll get absolutely no benefit from this custom integration. All the new features are only in the newer implementation.

I could have added newer YAML configuration support (e.g., allow it to be configured under a “google_maps” key), but that is a LOT of extra work, so I decided not to bother. I have that kind of support in most of my other custom integrations, so I know how much extra work is involved.

If you do use the new UI based config, and still leave the legacy YAML config in, that is ok. You’ll get two sets of entities. It does give you an opportunity to test out the new implementation (via the new entities), and slowly convert your system over at your leisure, should you decide to do so. If not, just delete the UI integration entries, uninstall the custom integration, and you system is just as it was before.

1 Like
69

So can we follow a person on a map ?