[Integration] Android TV live cam video streams in a Picture in Picture triggered by motion detection. AKA video-doorbell

@seanblanchfield

Thanks for beeing patient. I think I’m in deep water. It is very strange that it is not working when it is working for others.

image

// Copyright 2012 The Chromium Authors
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

/**
 * @fileoverview
 * NOTE: This file is deprecated, and provides only the minimal LoadTimeData
 * functions for places in the code still not using JS modules. Use
 * load_time_data.ts in all new code.
 *
 * This file defines a singleton which provides access to all data
 * that is available as soon as the page's resources are loaded (before DOM
 * content has finished loading). This data includes both localized strings and
 * any data that is important to have ready from a very early stage (e.g. things
 * that must be displayed right away).
 *
 * Note that loadTimeData is not guaranteed to be consistent between page
 * refreshes (https://crbug.com/740629) and should not contain values that might
 * change if the page is re-opened later.
 */

/** @type {!LoadTimeData} */
// eslint-disable-next-line no-var
var loadTimeData;

class LoadTimeData {
  constructor() {
    /** @type {?Object} */
    this.data_ = null;
  }

  /**
   * Sets the backing object.
   *
   * Note that there is no getter for |data_| to discourage abuse of the form:
   *
   *     var value = loadTimeData.data()['key'];
   *
   * @param {Object} value The de-serialized page data.
   */
  set data(value) {
    expect(!this.data_, 'Re-setting data.');
    this.data_ = value;
  }

  /**
   * @param {string} id An ID of a value that might exist.
   * @return {boolean} True if |id| is a key in the dictionary.
   */
  valueExists(id) {
    return id in this.data_;
  }

  /**
   * Fetches a value, expecting that it exists.
   * @param {string} id The key that identifies the desired value.
   * @return {*} The corresponding value.
   */
  getValue(id) {
    expect(this.data_, 'No data. Did you remember to include strings.js?');
    const value = this.data_[id];
    expect(typeof value !== 'undefined', 'Could not find value for ' + id);
    return value;
  }

  /**
   * As above, but also makes sure that the value is a string.
   * @param {string} id The key that identifies the desired string.
   * @return {string} The corresponding string value.
   */
  getString(id) {
    const value = this.getValue(id);
    expectIsType(id, value, 'string');
    return /** @type {string} */ (value);
  }

  /**
   * Returns a formatted localized string where $1 to $9 are replaced by the
   * second to the tenth argument.
   * @param {string} id The ID of the string we want.
   * @param {...(string|number)} var_args The extra values to include in the
   *     formatted output.
   * @return {string} The formatted string.
   */
  getStringF(id, var_args) {
    const value = this.getString(id);
    if (!value) {
      return '';
    }

    const args = Array.prototype.slice.call(arguments);
    args[0] = value;
    return this.substituteString.apply(this, args);
  }

  /**
   * Returns a formatted localized string where $1 to $9 are replaced by the
   * second to the tenth argument. Any standalone $ signs must be escaped as
   * $$.
   * @param {string} label The label to substitute through.
   *     This is not an resource ID.
   * @param {...(string|number)} var_args The extra values to include in the
   *     formatted output.
   * @return {string} The formatted string.
   */
  substituteString(label, var_args) {
    const varArgs = arguments;
    return label.replace(/\$(.|$|\n)/g, function(m) {
      expect(m.match(/\$[$1-9]/), 'Unescaped $ found in localized string.');
      return m === '$$' ? '$' : varArgs[m[1]];
    });
  }

  /**
   * As above, but also makes sure that the value is a boolean.
   * @param {string} id The key that identifies the desired boolean.
   * @return {boolean} The corresponding boolean value.
   */
  getBoolean(id) {
    const value = this.getValue(id);
    expectIsType(id, value, 'boolean');
    return /** @type {boolean} */ (value);
  }

  /**
   * As above, but also makes sure that the value is an integer.
   * @param {string} id The key that identifies the desired number.
   * @return {number} The corresponding number value.
   */
  getInteger(id) {
    const value = this.getValue(id);
    expectIsType(id, value, 'number');
    expect(value === Math.floor(value), 'Number isn\'t integer: ' + value);
    return /** @type {number} */ (value);
  }

  /**
   * Override values in loadTimeData with the values found in |replacements|.
   * @param {Object} replacements The dictionary object of keys to replace.
   */
  overrideValues(replacements) {
    expect(
        typeof replacements === 'object',
        'Replacements must be a dictionary object.');
    for (const key in replacements) {
      this.data_[key] = replacements[key];
    }
  }
}

/**
 * Checks condition, throws error message if expectation fails.
 * @param {*} condition The condition to check for truthiness.
 * @param {string} message The message to display if the check fails.
 */
function expect(condition, message) {
  if (!condition) {
    throw new Error(
        'Unexpected condition on ' + document.location.href + ': ' + message);
  }
}

/**
 * Checks that the given value has the given type.
 * @param {string} id The id of the value (only used for error message).
 * @param {*} value The value to check the type on.
 * @param {string} type The type we expect |value| to be.
 */
function expectIsType(id, value, type) {
  expect(
      typeof value === type, '[' + value + '] (' + id + ') is not a ' + type);
}

expect(!loadTimeData, 'should only include this file once');
loadTimeData = new LoadTimeData();

// Expose |loadTimeData| directly on |window|, since within a JS module the
// scope is local and not all files have been updated to import the exported
// |loadTimeData| explicitly.
window.loadTimeData = loadTimeData;

console.warn('crbug/1173575, non-JS module files deprecated.');

VM306:182 crbug/1173575, non-JS module files deprecated.
(anonymous) @ VM306:182


Edit: So I had a spare camera. Same make, same model laying around. Plugged that into my main Lan. The other one is in a separate lan that is barred from the internet, but I can access it from my other lan.
This spare camera shows up in Chrome desktop, but not in chrome app on android. Still white screen.

Could it be the ports? How to I tell webrtc which ports to use?

There is nothing in the javascript console screenshots your posted that seem relevant to the issue you are experiencing. However, your theory that the whitescreen could be related to your network subnets and port forwarding sounds plausible. If the webRTC session that the HTML page tries to start fails due to network connectivity, then I suppose it might simply not load anything and leave the page blank.

Since I originally wrote my guide, webrtc-camera v3 has come out, and it’s a significant improvement. But it works a bit differently. The answer to your question depends on whether you are using v3 or v2.5.

  • If you are using v3, then webrtc-camera is connecting back to a “go2rtc” proxy server, which either it is running for you, or which you have set up yourself somewhere. That uses 8555 for webrtc communication. To talk to it from the internet, you need to port forward udp/tcp 8555 to it. It sounds like you have multiple subnets with firewall rules between them also, so you’re on your own in terms of making sure that traffic to 8555 has a clear run across those network segments.

  • If you are using webrtc-camera v2.x, then it is running RTSP2WebRTC as the stream proxy server, and you need to forward ports 50000/udp - 51000/udp as described here, and also make sure that any firewalls or other rules controlling traffic between your subnets isn’t going to block it.

You were right. Due to that the cam, homeassistant and nvidia were on different subnets, it wouldn’t stream right.
Thanks for all the patience and help. Will now see if it is possible to open ports between the subnets

Anyone knows all the ports? 8555, 544 and 443?

got 3 vlans. home assistant is on main lan, shield on iot and cam on camlan. using the v3 of webrtc.

Edit: For those with same problem. I opened port 8123 from IOT vlan to LAN vlan. Blocked Cam Vlan from LAN and IOT, but opened LAN to CAM vlan and IOT vlan. This way the IOT and CAM is blocked from LAN and eachother. But LAN can see iOT and CAM. IOT can see LAN through port 8123. Using unify.

1 Like

It appears that the link to the updated version is broken.

Is it worth trying this with the pre-update instructions ?

Thanks for letting me know about the broken link in my blog post. I have updated it to fix the link, so it directs you to the updated version of the post that describes it all working with go2rtc. I recommend you read the updated version only.

I have followed the new guide and I have things working almost perfectly. The only remaining oddity is that if I send a text message like in the post.json the popup works great and I can continue watching TV, etc. However, if I send my camera url the background goes black as soon as the video starts playing and I am left on a blank screen. I have to re-select input to get back to what I was doing. Sony Bravia. Same behavior with both APKs btw. Any suggestions appreciated.

Interesting change in behavior. While I was working on the black screen/changing input issue, I updated home assistant to operating system 10.3 and the behavior of this malfunction changed completely. The force change of input went away. Unfortunately, the new behavior seems worse. I can only see the video in the popup if I am on a home screen or if the video is paused(google tv and firestick). (My mac mini input just shows a green box for the video.) I can hear the sound. No related errors in the logs that I can find.

I got it working as per @ seanblanchfield’s blog post. The only strange thing is that, when the stream starts (ie, after the player fully loads) the TV sound stops. I’m not sure if this is related to the fact that my TV output is a Sonos arc (connected through HDMI arc - just realised the naming), but to get it back I need to force kill whatever was playing (for example Netflix) and then resume the video.

I can work around by playing the stream directly from go2rtc and using the webrtc.html page instead of the stream.html page, but I was wondering if anyone else has faced this mystery.

Yes, that’s because of the audio. If you’d add the no audio option to your camera config it runs fine.

Yeah, with the webrtc.html go2rpc page, with ?media=video, the pipup app never steals the audio, so everything works.

Are you doing that through the HA camera short-lived link? If yes, could you please state what the params are? I’ve got it sorted, but other folks may want/need to use the HA link instead, so it could be useful to leave the info out there.

no i don’t I run a reversed proxy

I have followed your instruction but i can’t get the frigate video to load. Any idea what i am doing wrong?

changed the code? What if you paste the URL in a browser, does that work?

I am using the below script to test. The web page doesn’t work. I am running Frigate as an addon. should i change the address to something else?

- service: rest_command.pipup_url_on_tv
  data:
    title: Driveway
    message: Live Stream of Driveway
    width: 360
    height: 200
    url: 'http://192.168.86.93:5000/api/driveway'

Yes, to a working camera url.

That’s what i am trying to figure out. I am unable to find the frigate link to a specific camera.

Mostly that’s http://haipaddress:5000/api/cameraname where you repalce haipaddress with the actual ip address of your ha instance and cameraname with the name you gave your camera, but this is very specific to an environment. So don’t think anyone can help you with this. I also made the assumption you run your frigate as addon on hassio and selected port 5000 (and have it enabled as such in the configuration part of the addon).

1 Like

Port 5000 was disable by default. I enabled it and the link is working now. Thanks for your help.

2 Likes

Tip:
Add these lines in the automation. I see that alot are having the same issue that PipUp are not running after a while.

  • service: androidtv.adb_command
    data:
    command: >-
    ps -ef | grep -v grep | grep pipup || adb shell am start
    nl.rogro82.pipup/.MainActivity
    target:
    entity_id:
    - media_player.nividia_shield
    - service: androidtv.adb_command
    data:
    command: input keyevent KEYCODE_WAKEUP
    target:
    entity_id:
    - media_player.nividia_shield

I’m stugging to get it to work - the temporary URL (captured from trace) works fine externally or internally in any browser but not on the AndroidTV.
I’m using the built in go2rtc with the WebRTC Plugin. HA is hosted in docker in Synology.

service: rest_command.pipup_url_on_tv
data:
  title: Door
  message: Someone is at the front door
  width: 640
  height: 480
  url: https://<mydns>/webrtc/embed?url={{ link_id }}
  duration: 60

I’m using the apk from Sean’s guide.

Any idea?