Plea to split companion app in 2 separate apps, one front end, one backend (and with MQTT/REST support)

[R5fan]

I dont know who to beg for this, so I will post it here.

I currently have 2 major problems with HA, and they both have the same root cause and the same (to me, obvious) solution. My first issue is with my wall mounted tablets, that I struggle to keep on when needed and off when not needed. Using command notifications to keep a screen on, and then have it time out, is often just too slow and unreliable. Requiring a cloud round trip to a third party for something like this seems completely counter to anything HA stands for. We hack firmwares on 500 euro devices to gain local control, but we cant get local control over our frontend?

My second issue is a year old bug that camera streams (sometimes?) keep streaming in the background, even after force closing companion, which has gobbled up so much bandwidth twice now, that I exceeded my data plan twice and got a 50 euro surcharge on top. Because force closing did not help, the only “solution” I had for this was turning off mobile data (!) until I replaced all my streams with blue iris iframes.

To me its very clear that companion does two very different things; it provides a GUI (basically a browser) and it provides sensor data + limited API. The latter should always run in the background or it does not work. The former should never run in the background (and most certainly not keep video streams active).

Can we please have 2 app for this, with different permissions? Or just strip the front end out of companion, and make it only do sensor + commands. We can use any browser for the frontend anyway.

And for the backend part, can we please please please get MQTT or HTTP REST support, so I dont need internet access and send google notifications across the globe, that may or may not be received by the tablet 4 meters from my server, and do that every x minutes just to keep my screen on?

I know wallpanel does this, and I keep going back and forth between wallpanel and companion, but wallpanel suffers from the same streaming in the background issue (probably the same browser component?) and it doesnt have the wealth of sensor data that companion provides

[HasQT (Has)]

Fully Kiosk: Launch local Android apps from your HA dashboard

Fully Kiosk is a great solution for wall-mounted tablets running Lovelace dashboards. Alongside Home Assistant, it can be useful to run local Android apps, for example to control a device which is not yet integrated into Home Assistant. There are a few different methods of launching local Android apps from a Lovelace dashboard running in Fully Kiosk. You can use Android intents, the Fully Kiosk JavaScript API or the Fully Kiosk REST API. JavaScript and REST API docs are here. Both APIs are Plu…

Sort of sounds like this might be what you want for your wall tablets. Has some interesting ways of turning the screen on etc.

Sounds like you have the wrong addresses set for a local tablet too, or no local address set in the companion app.

[R5fan]

There are alternatives for the front end, like fully kiosk or wallpanel, but they do not provide the backend/sensor functionality of companion.

Sounds like you have the wrong addresses set for a local tablet too, or no local address set in the companion app.

I only have a local IP address set on the tablets, but Im not sure which issue you are referring to here? command notifications not arriving or the camera streaming issue (which isnt really a big issue when used locally), I dont see how either would be related to the wrong address?

[dshokouhi]

@R5fan

My first issue is with my wall mounted tablets, that I struggle to keep on when needed and off when not needed. Using command notifications to keep a screen on, and then have it time out, is often just too slow and unreliable. Requiring a cloud round trip to a third party for something like this seems completely counter to anything HA stands for. We hack firmwares on 500 euro devices to gain local control, but we cant get local control over our frontend?

if your notifications are too slow you should read the docs on how to speed them up so they show up almost instantly. Read into critical notifications. The main goal of the companion app is to run on your cell phone when you are not at home and in order to receive notifications we use firebase for that. There are some features that tablet users may benefit from but that is not the primary focus.

@R5fan

My second issue is a year old bug that camera streams (sometimes?) keep streaming in the background, even after force closing companion, which has gobbled up so much bandwidth twice now, that I exceeded my data plan twice and got a 50 euro surcharge on top. Because force closing did not help, the only “solution” I had for this was turning off mobile data (!) until I replaced all my streams with blue iris iframes.

simple solution, remove any and all cameras off the homescreen and be diligent when you leave the app not to leave a camera in view. This issue also impacts chrome for android. You also mention that even wallpanel suffers, so yes a bad bug but if you read the actual issue on github you would’ve seen all of the workarounds mentioned.

@R5fan

The latter should always run in the background or it does not work. The former should never run in the background (and most certainly not keep video streams active).

our sensor worker does indeed run in the background, you are talking about a known issue with the camera and its one that the team has been struggling to fix. If you know how to fix please come and help or at least take a look at the history of the issue to see the team is trying to solve it. Its not like we want that bug there.

@R5fan

Can we please have 2 app for this, with different permissions? Or just strip the front end out of companion, and make it only do sensor + commands. We can use any browser for the frontend anyway.

No the entire purpose of the app is to view the frontend and also get data off the phone, it is a companion app after all.

@R5fan

And for the backend part, can we please please please get MQTT or HTTP REST support, so I dont need internet access and send google notifications across the globe, that may or may not be received by the tablet 4 meters from my server, and do that every x minutes just to keep my screen on?

the app makes use of the HA REST API for a lot of data and we also use webhooks for sending data back home. Google is used for firebase notifications and unless you can help us to code up an alternative I think you need to understand how difficult it is and why firebase makes sense because its already there and easy to implement for a developer. Also read the docs on how to speed up notifications.

[R5fan]

if your notifications are too slow you should read the docs on how to speed them up so they show up almost instantly.

My complaint about notifications that are slow and unreliable are for command notifications, AFAICS we have no control over the channel or their importance or any other way to fix this. Nor should we need to, because I insist this is a very wrong way to send commands to a HA client, particularly to a dedicated HA tablet thats on wifi and within arms reach of its HA server.

simple solution, remove any and all cameras off the homescreen

Thats 80% of the functionality of my front end. Of course we can create a custom view with nothing on it just for companion (and pray it doesnt load any of the others in the background) while using a separate browser to actually use the front end, but how is that better than having “companion sensors app” (and if you must, a companion browser)?

and be diligent when you leave the app not to leave a camera in view.

Does not reliably prevent the problem.

This issue also impacts chrome for android.

I dont know, I dont use chrome. I can not reproduce the problem on firefox or brave browser. Being able to pick a browser that respects background data limits is only possible if you untangle the front from the backend though.

If you know how to fix please come and help

Well, I kinda told you how you can fix it Remove the browser from companion, let users pick their own browser that may or may not be bugged. At least we wouldnt be blaming you for it then.

the app makes use of the HA REST API for a lot of data and we also use webhooks for sending data back home.

Then why not extend that so it goes both ways, so we can send instructions to companion without having to suffer firebase? You say its not a focus, but how many HA users do you think use tablets at home to control it? If its not a majority, it wont be far off.

Anyway, thanks for making clear its not gonna happen, so I might as well start standardizing on fully kiosk and wallpanel and finding ways to work around their missing sensors. Goodbye companion… sniff.

[dshokouhi]

@R5fan

My complaint about notifications that are slow and unreliable are for command notifications, AFAICS we have no control over the channel or their importance or any other way to fix this. Nor should we need to, because I insist this is a very wrong way to send commands to a HA client, particularly to a dedicated HA tablet thats on wifi and within arms reach of its HA server.

If you had taken just a moment to READ the docs you would understand how to get a notification to show up immediately. Because you are unwilling to spend the time to research your issue I will link you directly to what you need:

Critical notifications | Home Assistant Companion Docs

@R5fan

Thats 80% of the functionality of my front end. Of course we can create a custom view with nothing on it just for companion (and pray it doesnt load any of the others in the background) while using a separate browser to actually use the front end, but how is that better than having “companion sensors app” (and if you must, a companion browser)?

There is nothing stopping you from using another browser and using the app for the other features. I am just telling you how to avoid the bug which is present in chrome. You just need to be diligent in the state that you leave the app.

@R5fan

Being able to pick a browser that respects background data limits is only possible if you untangle the front from the backend though.

I dont understand what your goals for “untangling” are. There is nothing stopping you from using the app for its background features and simply not using the frontend portion of the app. The app uses authenticated webview as documented from HA

Authenticated Webview | Home Assistant Developer Docs

if you dont want to use this nothing is forcing you from using it.

@R5fan

Remove the browser from companion, let users pick their own browser that may or may not be bugged. At least we wouldnt be blaming you for it then.

no need, just use the features you want.

@R5fan

Then why not extend that so it goes both ways, so we can send instructions to companion without having to suffer firebase?

Thats a lot easier said than done but we are open source so why not spend some free time, learn android and add the feature so everyone can benefit? If you spent some time reading docs then you wouldnt be suffering due to firebase. My notifications show up within a second irregardless of location. Also keep in mind an open connection to firebase is a lot more battery friendly than maintaining an open websocket or MQTT connection as you would like to describe. Majority of users are on a cell phone and not on a wall mounted tablet where battery is not of concern.

[R5fan]

@dshokouhi

If you had taken just a moment to READ the docs you would understand how to get a notification to show up immediately.

I wish. I admit I wasnt using priority or ttl for my command notifications, I did read that and even use it for my phone, but i assumed that was just for general notifications -no hint is given in the command notification page, if it did help waking up a sleeping tablet it might have been worth repeating there; but it doesnt, for me at least; I just tried. I just sent a dozen notifications of all kinds, text and commands, with and without priority and TTL. Even with the tablet next to me, screen on, companion in the foreground connected to and interacting with HA. Nothing. Notification log in companion shows nothing recent. Then all of a sudden, 10 minutes later I get 10 notifications within a second. It seems to work better on my phone (which has a more recent android version, not sure thats related, phone is on android 11, tablet on 9), but I dont need to send commands to my phone to keep the screen on.

There is nothing stopping you from using another browser and using the app for the other features.

Other than my fear of using up my data cap while walking the dog, especially if god forbid I accidentally where to click on an installed app or recently opened it to change some sensor settings.

Anyway, to each their own, but Im not going to run companion + wallpanel or fully kiosk on my tablets and companion + firefox on my phone and then try not to get confused which app I need to pull sensor data from and what app and method I need to send commands to for which device. Wallpanel doesnt solve the background streaming issue on my phone either, so there is no single solution for me, but at least for my tablets it does 95% of what I need without relying on firebase notifications to keep the screen on or issue TTS alerts. So its less hassle to standardize on that and work around the missing 5%

Also keep in mind an open connection to firebase is a lot more battery friendly than maintaining an open websocket or MQTT connection as you would like to describe. Majority of users are on a cell phone and not on a wall mounted tablet where battery is not of concern.

Im guessing close to no one issues screen_on commands to their phones though? Im not suggesting to drop firebase notifications, clearly that has its uses, I just dont think keeping a screen on for a local tablet should be done that way and with no alternative.

[NathanCu (Nathan Curtis)]

@R5fan

Im guessing close to no one issues screen_on commands to their phones though?

You’re probably right - also - The rest of us just use Fully, and connect to the internally accessible IP address for HA and forget about it to avoid the rest of your concerns about caps.

[dshokouhi]

@R5fan

Then all of a sudden, 10 minutes later I get 10 notifications within a second. It seems to work better on my phone (which has a more recent android version, not sure thats related, phone is on android 11, tablet on 9), but I dont need to send commands to my phone to keep the screen on.

Check that google play services is up to date, this sounds more like a device issue than an app issue. Especially considering that you mention it works on one device and not another. You can also check that the app has permission to run in the background.

@R5fan

I just dont think keeping a screen on for a local tablet should be done that way and with no alternative.

Just because a feature exists doesn’t mean that an alternative must be offered at the same time, think about the user that requested the feature and is happy that the request was implemented. You are asking for a lot of work to be done and asking that the volunteers focus their time on that. The whole point of being a volunteer is to work on whatever it is that they want.

[R5fan]

@NathanCu

y, and connect to the internally accessible IP address for HA and forget about it to avoid the rest of your concerns about caps.

Im confused. Havent really used fully kiosk yet, I assume it gives control over the screen, but does it also provide tablet/phone sensor data (over MQTT or otherwise) for things like location, battery, alarm, detected activity, phone status and some API for TTS notifications? Does it prevent background camera streams ? If not, it still isnt a complete alternative to wallpanel or companion. If it does, then I should definitely check it out.

[NathanCu (Nathan Curtis)]

with the exception of your background streams - you’ll probably have to do something about that on your own with good design… Notice I don’t have any of my cameras on my main landing page - for that exact reason) and please excuse my janky redaction of location and persons.

That’s the fully management page from the tablet itself. There’s literally PAGES of settings.

Then it exposes TONS of entities in HA:

The solution is rarely to change a tool, the solution is usually to use the correct tool.

[R5fan]

this sounds more like a device issue than an app issue.

Maybe, but delayed android notifications are pretty much a fact of life. And for sure they dont work without cloud access.

You can also check that the app has permission to run in the background.

It does, but the app was also very much running in the foreground, so thats not likely it. It also not like its exclusive to this tablet, or HA, Ive had doorbell notifications arrive on my phone 15 minutes after the event. It just happens. There is a reason I dont want my doorbell to depend on a cloud service when Im actually at home within spitting distance of the device. I smashed my ring doorbell for that reason (and a few others). The same logic applies here.

Just because a feature exists doesn’t mean that an alternative must be offered at the same time, think about the user that requested the feature and is happy that the request was implemented. You are asking for a lot of work to be done and asking that the volunteers focus their time on that. The whole point of being a volunteer is to work on whatever it is that they want.

I get that. I also get that I can always ask my money back if Im not happy. Its not that I dont appreciate the work thats being done, its that I think what I suggest could make it better, and may even reduce the amount of work. Im basically suggesting you scrap half the app, so users can pick whatever existing browser that works best for them. Saves you from trying to solve a bug that might even be out of your control; If chrome suffers the same background streaming bug, just recommend they use firefox or brave or opera (is that still a thing?), or whatever else that does not suffer from it. There is no lack of browsers on any platform. There is a lack of good HA “companion” apps that can feed HA with useful device and sensor data and provide it with some API for commands (screen, media, TTS, etc). Ideally with a choice of not putting your data cap at risk.

[dshokouhi]

@R5fan

Saves you from trying to solve a bug that might even be out of your control;

Anything related to HA frontend goes directly to HA frontend, we don’t look into those issues because the frontend is just a webview.

We also provide features in our webview that enhances the HA frontend so you can do things like launching any app on your device from a lovelace card. We also provide haptic feedback support based on androids haptic system. There is also a swipe gesture to quickly get to the quick bar. So there are some benefits for it. The webview portion of the app also makes use of google exoplayer which is needed for H265 video playback. So it has its uses and people are happy with it. Some of these are simply not possible in chrome and are easily handled by the app which is why we do these things.

Android WebView | Home Assistant Companion Docs

You dont have to use the frontend part of the app if you dont want to but there won’t be any scraping of work already done because a user can use any browser. The webview portion of the app is also critical for HA setup as a lot of users use the app to setup an HA server for the first time after plugging things in. I am sure you have noticed the link to the both the android and iOS app when you first install HA as the apps are a way to get going. Webview is very much important here especially for new users.

[R5fan]

@dshokouhi

Anything related to HA frontend goes directly to HA frontend, we don’t look into those issues because the frontend is just a webview.

Issues like camera streams not closing? If that is due to webview, which may well be the case, then its basically a huge bug you can not solve. Except by offering at the very least the option to not use it.

We also provide features in our webview that enhances the HA frontend so you can do things like launching any app on your device from a lovelace card.

Im no developer, but AFAIK you dont need “your own browser” for that. You can call a URI scheme from javascript and it should work on any browser. browser-mod also enables this, and it doesnt require webview, it works on any browser.

We also provide haptic feedback support based on androids haptic system.

Ok, no experience with this. Doesnt sound like a big deal to me though, Im certain I could live without it.

There is also a swipe gesture to quickly get to the quick bar.

IM sure there are dozens of people who use that on their phones. Actually, Im not.

The webview portion of the app also makes use of google exoplayer which is needed for H265 video playback.

Well, Im sorry, but most users I know off that (like me) are heavily dependent on video, feel forced to revert to mjpeg (!) or something like blue iris UI3 iframes because there is no other viable solution to stream even h264 without excessive lag, instability and/or inability to stream to multiple devices. So h265 support sounds great, but Im not holding my breath for workable h264 support and until that happens, Im sure we can all live without h265. And as long as streams are not closed reliably, I dont even care what codec it uses.

You dont have to use the frontend part of the app if you dont want

But I do. The front end loads whenever I launch companion, and it doesnt close when i close companion. At the very least consider an option to disable the frontend/webview part. That wont solve the lack of MQTT or local device control, but at least it would allow using companion only for its sensors with no worries and allow us to use something else for the frontend.