[Integration] Android TV live cam video streams in a Picture in Picture triggered by motion detection. AKA video-doorbell

You might be able to add a condition checking the state of the android box before calling the command.

Posted an updated APK and created a quick write up how I am using it.

1 Like

Thanks @DesertBlade. Your new APK (v1.1) works for me with JavaScript and WebRTC.

1 Like

Awesome to hear! Since my latest build I have not had an issue with the white video box loading. Hoping to spend some more time and see if there are any other tweaks to help with performance.

1 Like

Thank you. Your option is working. Everything that was before you didn’t work, but your version worked perfectly

1 Like

I have created detailed instructions on how to install and run PiPup, this can help those who do not understand how to do it.

2 Likes

I found out that with AgentDVR you can output a video stream to a PiPup. I added to this post how it can be done. AgentDVR is a smart server with built-in WebRTC. Video stream without delay

I’ve just followed Sean’s guide and managed to get pipup responding from a curl command.

When triggering the script I get an error page instead of the video feed: (net::ERR_CLEARTEXT_NOT_PERMITTED)

I’m trying to use a Nest Hello doorbell (An old wired RTSP device) which is added with the Nest integration:

rest_commands.yaml:

pipup_url_on_tv:
  # Use with Webrtc camera as described here:
  # https://github.com/AlexxIT/WebRTC/wiki/Cast-or-share-camera-stream#html-page
  url: http://192.168.86.200:7979/notify
  content_type: 'application/json'
  verify_ssl: false
  method: 'post'
  timeout: 20
  payload: >
    {
      "duration": {{ duration | default(20) }},
      "position": {{ position | default(0) }},
      "title": "{{ title | default('') }}",
      "titleColor": "{{ titleColor | default('#50BFF2') }}",
      "titleSize": {{ titleSize | default(10) }},
      "message": "{{ message }}",
      "messageColor": "{{ messageColor | default('#fbf5f5') }}",
      "messageSize": {{ messageSize | default(14) }},
      "backgroundColor": "{{ backgroundColor | default('#0f0e0e') }}",
      "media": { 
        "web": {
          "uri": "{{ url }}", 
          "width": {{ width | default(640) }},
          "height": {{ height | default(480) }}
        }
      }
    }

scipts.yaml

display_front_door_pip_popup_on_tv:
  alias: Display Driveway PIP Popup on TV
  mode: single
  variables:
    link_id: '{% for _ in range(40) %}{{ range(10)|random }}{% endfor %}'
  sequence:
  - service: webrtc.create_link
    data:
      link_id: '{{ link_id }}'
      entity: camera.front_door
      open_limit: 1
      time_to_live: 60
  - service: rest_command.pipup_url_on_tv
    data:
      title: Door
      message: Someone is at the front door
      width: 640
      height: 480
      url: http://192.168.86.50:8123/webrtc/embed?url={{ link_id }}&webrtc=false

Any idea what I’ve done wrong?

what pipup apk are you using? typically pipup requires HTTPS ( [DesertBlade’s supposedly disables that but I haven’t tested personally). The error message though is telling you exactly that. I’m guessing you have the other pipup installed.

Make sure you installed this one:

EDIT: Just tested @DesertBlade’s apk and yes it works without https through go2rtc just fine. That helps so I don’t need to expose my cameras externally through a https address. Thanks.

1 Like

@calisro Is correct. I believe in Android 8 apps had to use HTTPS url calls. If the application needed regular HTTP traffic they had to set android:usesCleartextTraffic=“true” in the manifest.

Anything that is being accessed over the public internet should have a signed certificate and you should be using HTTPS. If everything is staying internal to your network your risk of exposure is low and setting up self-signed certificates locally is usually more hassle than its worth.

I didn’t realize Nest now has local streams. I helped built NST Manager for Smartthings years ago and the workaround then was to make your stream public.

Using DesertBlades PiPup I’m getting an error stating “ERROR: Can’t get URL for camera.frontdoor”.
Would this mean my issue is with webRTC?
I’m using the following script:

	Dorbell_TV:
	  alias: Display Driveway PIP Popup on TV
	  mode: single
	  variables:
	    link_id: '{% for _ in range(20) %}{{ range(10)|random }}{% endfor %}'
	  sequence:
	  - service: webrtc.create_link
	    data:
	      link_id: '{{ link_id }}'
	      entity: camera.frontdoor
	      open_limit: 1
	      time_to_live: 60
	  - service: rest_command.pipup_url_on_tv
	    data:
	      title: Door
	      message: Someone is at the front door
	      width: 640
	      height: 480
	      url: http://mylocalip/webrtc/embed?url={{ link_id }}&webrtc=false

I’m pulling my limited hair out trying to figure this out.

Instead of using “camera.frontdoor” try connecting to the RTSP stream directly.

Remove :

entity: camera.frontdoor

add:

url: rtsp://rtsp:[email protected]:554/.....

The rtsp stream is very dependent on the camera.

More info: Cast or share camera stream · AlexxIT/WebRTC Wiki · GitHub

Hi calisro,

When you call the Piup app on the Android Tv via the adb command (to make sure it is running and reachable for notifications), does it also start the application which causes other running apps to pause/close?

Is it not possible to just start tje app in the background?

Rarely but sometimes it’ll interfere.

Anyone experiencing that the stream crashes on Nvidia Shield?

My Shield is wired and I don’t have any speed issues on the network, but sometimes when HA triggers Pipup with a cam feed, it either shows for a few seconds and then disappear or it simply just stutter the current app for a few seconds and then “crashes” ( doesn’t popup).

Hi PPL! Thanks for all the great work around this idea.
Let me clarify some things to help to achieve this works in any setup.

PIP setup:
The play store version published only supports HTTPS. So, if your setup don’t have a public and valid certificate, the TV cant display the cam video stream.
There is a modified one in a fork, that allows to use HTTP. This one can be found HERE
This app needs to be pushed via ADB (every TV model have a way, but always is around enabling “developer options” in android system, allow “unknown sources”, etc etc) because is not published in the play store.

Stream directly from cam vs camera proxy:
This is an important thing to avoid delays. The ideal is to send video directly from source to destination (in this case, camera stream to TV) but, there is some limitation in the video player of the android systems.
They need the stream with specific codec setup. You can notice this “incompatibility” because your HA dont catch audio from the stream source of your cam. Thats why, sometimes, you need to transcode it in the HA server you use (in my example i setup it inside a tablet, but any kind of setup can be used)
It depends of the kind of stream your cameras generates. In TP-LINK TAPO case, the stream is not directly compatible with PIP app, so, need to use camera proxy of HA to convert it to an PIP app readeable stream.
Obviously, this increases the latency and stream gets a delay of “live view”, but it is a doorbell, is not critical.
Resuming, if you can access to your camera stream with something like “http(s)://user:[email protected]:(port)/stream1” ie, you can do PIP app to read it directly. But, if you uses something like “rtsp://” PIP app its not able to read it directly (its an android limitation of the standard web player it integrates), so, you need to use the proxy of HA yo transcode the stream.

Thanks @seanblanchfield for the complete analysis/documentation you generate. Great work!

Hope this info helps to understand how the whole thing works.

Greets

4 Likes

Hi Sean. Trying to follow your instructions.
Got the camera working

Got the notifiction working on my Nvidia shield. Did the “curl -d “@post.json” -H “Content-Type: application/json” -X POST …” from my raspberry pi and a picture popped up on the screen.

Opened port 50000-500010 on my router. But I don’t know what you mean by. “told the WebRTC integration that these ports were available for it to use”
image

When I try to call the script now. There are only a white notification that pops up with the message that someone are at the door. I belive that it is because my ports is not working.

display_ytterdor:
  alias: Ytterdør pip
  mode: single
  variables:
    link_id: "{% for _ in range(40) %}{{ range(10)|random }}{% endfor %}"
  sequence:
    - service: webrtc.create_link
      data:
        link_id: "{{ link_id }}"
        entity: camera.ytterdor
        open_limit: 1
        time_to_live: 60
    - service: rest_command.pipup_url_on_tv
      data:
        title: Door
        message: Someone is at the front door
        width: 640
        height: 480
        url: https://xxxxxx.duckdns.org:8123/webrtc/embed?url={{ link_id }}&webrtc=false

image

Edit: It is not the ports. I can see the stream from outside LAN.

Tried this under “Developer Tools > Services”

service: rest_command.pipup_url_on_tv
data:
  title: Doorbell
  message: There's somebody at the door
  url: rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4

Got this message:

Tried this:

And that worked.

I have no idea right now what the white screen is. I’m not sure from your edits whether that’s still happening. If it does happen again, you could attempt to analyse it by visiting the relevant link in a desktop browser, and checking the javascript console and developer tools for obvious errors.

I have an idea about the error message though. I can only depend on google translate for norweigian and I understand it to say “The website could not be loaded because of unknown URL scheme”. The problem is that you can only send https links to the pipup_url_on_tv command. All that PipUp is doing in this case is opening the URL in a webview browser session, and browsers cannot directly open RTSP streams.
You must change the link from
rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4
to something that starts with HTTPS. That’s why the Youtube URL is working.

The /webrtc/embed?url={{ link_id }} page provided by the webrtc camera integration is for exactly this purpose. I see you are already using it. It returns a HTML page with javascript that starts a WebRTC session that displays a video stream proxied from the original RTSP stream (as opposed to attempting to directly display a RTSP stream, which wouldn’t be supported by the browser on your android box).

I think the “nettsiden pa kunne ikka lastes inn fordi” error is a red herring, and you need to go back to figuring out why the /webrtc/embed?url={{ link_id }} page wasn’t displaying correctly. Start by seeing if it displays correctly when you visit in in a normal browser, and if not, check the HTML source, javascript console and so on. Perhaps the issue is as simple as the “link_id” having expired.

So I changed the creation of the url from random to a static “test”.
Pasted it in to Opera browser: https://xxxx.duckdns.org:8123/webrtc/embed?url=test&webrtc=false

display_ytterdor:
  alias: Ytterdør pip
  mode: single
  variables:
    link_id: "test"
  sequence:
    - service: webrtc.create_link
      data:
        link_id: "{{ link_id }}"
        entity: camera.ytterdor
        open_limit: 1
        time_to_live: 60
    - service: rest_command.pipup_url_on_tv
      data:
        title: Door
        message: Someone is at the front door
        width: 640
        height: 480
        url: https://xxxx.duckdns.org:8123/webrtc/embed?url={{ link_id }}&webrtc=false

Got this message.
image

Maybe the camera is setup wrong. Got my cameras plugged into Blue iris and I get a url from that.
image

Edit:
Now I got the url running:


But still the white screen on Nvidia Shield.

When I call the service and it creates the url, the box is white and I can use the url on my browser and it will stream the camers. This is telling me that the shield never got to stream it because if it could I wouldn’t been able to open it on my browser due to the open_limit is set to 1.


image

I don’t have an answer, but here’s some ideas:

  • Are you 100% sure that the Shield and your desktop browser are displaying exactly the same URL (the one with the “test” link_id?)
  • Can you try it without the &webrtc=false?
  • try calling the webrtc camera webrtc.create_link with an open_limit of 0 so that its unlimited, to rule out the limit somehow being hit before the Shield has a chance to try to load it.
  • Can you try to view the link on chrome on another android device?

From your posts, it sounds like PipUp is working and can display webpages with video streams on your Shield, and that webrtc-camera is working and can serve streams you can view from a desktop browser. I (and I think many others) can attest to webrtc-camera streams generally working on shield, so that seems to rule out weird codec incompatibilities. I think that makes it likely that the problem is a hard-to-see mistake in the URL you are sending to PiPUp, or something to do with the open_limit.