[Integration] Android TV live cam video streams in a Picture in Picture triggered by motion detection. AKA video-doorbell

I started getting this too. Which is weird since I’ve had this code running for a long time. Anyone know why?

And its even called out here:

Found out my issue: There are different versions of the apk available:
The original apk from Rogro82 - This one didn’t work for me.
The forked version by gmcmicken to support web-javascript - This is the one worked.
The forked version by desertblade to support http/Frigate - This one didn’t work for me.

I followed seanblanchfield excellent instructions & have gotten this to mostly work on my Sony XBR-65X930E TV. When the TV is playing from the cable box, it works as expected. However, if I’m playing a Playstation 4 game & the PIPUp activates, it causes the TV’s Android OS to reboot. I’m just wondering if anyone else has experienced this or better yet, a way to fix this? All video sources (cable, PS4 & a few others) go to a Yamaha AVR then, one HDMI cable from the AVR to the TV. As a work around, my automation currently doesn’t trigger if the PS4 is on. Thanks for any insight into this.

The version I made doesn’t have web/javascript. I struggled with getting the remote stream to work reliably.

I ended up using Frigates local MJPEG streams which loaded faster and had smoother playback than using the javascript WebRTC player.

You might be able to add a condition checking the state of the android box before calling the command.

Posted an updated APK and created a quick write up how I am using it.

1 Like

Thanks @DesertBlade. Your new APK (v1.1) works for me with JavaScript and WebRTC.

1 Like

Awesome to hear! Since my latest build I have not had an issue with the white video box loading. Hoping to spend some more time and see if there are any other tweaks to help with performance.

1 Like

Thank you. Your option is working. Everything that was before you didn’t work, but your version worked perfectly

1 Like

I have created detailed instructions on how to install and run PiPup, this can help those who do not understand how to do it.

2 Likes

I found out that with AgentDVR you can output a video stream to a PiPup. I added to this post how it can be done. AgentDVR is a smart server with built-in WebRTC. Video stream without delay

I’ve just followed Sean’s guide and managed to get pipup responding from a curl command.

When triggering the script I get an error page instead of the video feed: (net::ERR_CLEARTEXT_NOT_PERMITTED)

I’m trying to use a Nest Hello doorbell (An old wired RTSP device) which is added with the Nest integration:

rest_commands.yaml:

pipup_url_on_tv:
  # Use with Webrtc camera as described here:
  # https://github.com/AlexxIT/WebRTC/wiki/Cast-or-share-camera-stream#html-page
  url: http://192.168.86.200:7979/notify
  content_type: 'application/json'
  verify_ssl: false
  method: 'post'
  timeout: 20
  payload: >
    {
      "duration": {{ duration | default(20) }},
      "position": {{ position | default(0) }},
      "title": "{{ title | default('') }}",
      "titleColor": "{{ titleColor | default('#50BFF2') }}",
      "titleSize": {{ titleSize | default(10) }},
      "message": "{{ message }}",
      "messageColor": "{{ messageColor | default('#fbf5f5') }}",
      "messageSize": {{ messageSize | default(14) }},
      "backgroundColor": "{{ backgroundColor | default('#0f0e0e') }}",
      "media": { 
        "web": {
          "uri": "{{ url }}", 
          "width": {{ width | default(640) }},
          "height": {{ height | default(480) }}
        }
      }
    }

scipts.yaml

display_front_door_pip_popup_on_tv:
  alias: Display Driveway PIP Popup on TV
  mode: single
  variables:
    link_id: '{% for _ in range(40) %}{{ range(10)|random }}{% endfor %}'
  sequence:
  - service: webrtc.create_link
    data:
      link_id: '{{ link_id }}'
      entity: camera.front_door
      open_limit: 1
      time_to_live: 60
  - service: rest_command.pipup_url_on_tv
    data:
      title: Door
      message: Someone is at the front door
      width: 640
      height: 480
      url: http://192.168.86.50:8123/webrtc/embed?url={{ link_id }}&webrtc=false

Any idea what I’ve done wrong?

what pipup apk are you using? typically pipup requires HTTPS ( [DesertBlade’s supposedly disables that but I haven’t tested personally). The error message though is telling you exactly that. I’m guessing you have the other pipup installed.

Make sure you installed this one:

EDIT: Just tested @DesertBlade’s apk and yes it works without https through go2rtc just fine. That helps so I don’t need to expose my cameras externally through a https address. Thanks.

1 Like

@calisro Is correct. I believe in Android 8 apps had to use HTTPS url calls. If the application needed regular HTTP traffic they had to set android:usesCleartextTraffic=“true” in the manifest.

Anything that is being accessed over the public internet should have a signed certificate and you should be using HTTPS. If everything is staying internal to your network your risk of exposure is low and setting up self-signed certificates locally is usually more hassle than its worth.

I didn’t realize Nest now has local streams. I helped built NST Manager for Smartthings years ago and the workaround then was to make your stream public.

Using DesertBlades PiPup I’m getting an error stating “ERROR: Can’t get URL for camera.frontdoor”.
Would this mean my issue is with webRTC?
I’m using the following script:

	Dorbell_TV:
	  alias: Display Driveway PIP Popup on TV
	  mode: single
	  variables:
	    link_id: '{% for _ in range(20) %}{{ range(10)|random }}{% endfor %}'
	  sequence:
	  - service: webrtc.create_link
	    data:
	      link_id: '{{ link_id }}'
	      entity: camera.frontdoor
	      open_limit: 1
	      time_to_live: 60
	  - service: rest_command.pipup_url_on_tv
	    data:
	      title: Door
	      message: Someone is at the front door
	      width: 640
	      height: 480
	      url: http://mylocalip/webrtc/embed?url={{ link_id }}&webrtc=false

I’m pulling my limited hair out trying to figure this out.

Instead of using “camera.frontdoor” try connecting to the RTSP stream directly.

Remove :

entity: camera.frontdoor

add:

url: rtsp://rtsp:[email protected]:554/.....

The rtsp stream is very dependent on the camera.

More info: Cast or share camera stream · AlexxIT/WebRTC Wiki · GitHub

Hi calisro,

When you call the Piup app on the Android Tv via the adb command (to make sure it is running and reachable for notifications), does it also start the application which causes other running apps to pause/close?

Is it not possible to just start tje app in the background?

Rarely but sometimes it’ll interfere.

Anyone experiencing that the stream crashes on Nvidia Shield?

My Shield is wired and I don’t have any speed issues on the network, but sometimes when HA triggers Pipup with a cam feed, it either shows for a few seconds and then disappear or it simply just stutter the current app for a few seconds and then “crashes” ( doesn’t popup).

Hi PPL! Thanks for all the great work around this idea.
Let me clarify some things to help to achieve this works in any setup.

PIP setup:
The play store version published only supports HTTPS. So, if your setup don’t have a public and valid certificate, the TV cant display the cam video stream.
There is a modified one in a fork, that allows to use HTTP. This one can be found HERE
This app needs to be pushed via ADB (every TV model have a way, but always is around enabling “developer options” in android system, allow “unknown sources”, etc etc) because is not published in the play store.

Stream directly from cam vs camera proxy:
This is an important thing to avoid delays. The ideal is to send video directly from source to destination (in this case, camera stream to TV) but, there is some limitation in the video player of the android systems.
They need the stream with specific codec setup. You can notice this “incompatibility” because your HA dont catch audio from the stream source of your cam. Thats why, sometimes, you need to transcode it in the HA server you use (in my example i setup it inside a tablet, but any kind of setup can be used)
It depends of the kind of stream your cameras generates. In TP-LINK TAPO case, the stream is not directly compatible with PIP app, so, need to use camera proxy of HA to convert it to an PIP app readeable stream.
Obviously, this increases the latency and stream gets a delay of “live view”, but it is a doorbell, is not critical.
Resuming, if you can access to your camera stream with something like “http(s)://user:[email protected]:(port)/stream1” ie, you can do PIP app to read it directly. But, if you uses something like “rtsp://” PIP app its not able to read it directly (its an android limitation of the standard web player it integrates), so, you need to use the proxy of HA yo transcode the stream.

Thanks @seanblanchfield for the complete analysis/documentation you generate. Great work!

Hope this info helps to understand how the whole thing works.

Greets

4 Likes