SIP Doorbell, android tablet and HA integration

I have connected my Grandstream GDS3710 (a SIP enabled video door bell), a tablet running Android (and webrtc), using asterisk in between based on the doordroid.js card made available by @rdehuyss (Thanks a lot!!). I am now able to see and talk to the person ringing on the video door bell and opening the door (if I want to).

In the next posts I will describe what I have done to integrate my SIP door bell into home assistant using asterisk in the middle and with automations using node red.

Here we go. In order:

  • configuring asterisk
  • the home assistant part
  • the automations in node red.

Hope someone find this useful and will be able to adapt to its needs!!
GV

12 Likes
  1. Asterisk
    I am not going to describe all the installation of asterisk. There are quite a lot of examples available. I will only focus on the relevant parts
    In fact I am using freepbx which makes easier the configuration of asterisk using a nice gui.
    Create two PJSIP extensions for the GDS3710 and for the tablet. On the tablet, install gs wave (softphone from grandstream). The softphone will only be here to check whether your GDS and your tablet can talk to each other and that the freepbx configuration is correct.
    Once both extensions can create a connection, it means that the PJSIP setup for the GDS is OK.
    Let’s say that the GDS is extension 1001. The tablet for GS WAVE is 1002. Both are standard PJSIP extensions. Nothing fancy.

Create an additional 1003 virtual extension, I will explain why later.
Create a webrtc extension say 1004.

The configuration of the 1004 should look like this:

[1004]
type=endpoint
aors=1004
auth=1004-auth
tos_audio=ef
tos_video=af41
cos_audio=5
cos_video=4
disallow=h264,mpeg4
allow=ulaw,g729
context=from-internal
callerid=hass <1004>

dtmf_mode=rfc4733
aggregate_mwi=yes
use_avpf=yes
rtcp_mux=yes
ice_support=yes
media_use_received_transport=yes
trust_id_inbound=yes
media_encryption=no
timers=yes
media_encryption_optimistic=no
send_pai=yes
rtp_symmetric=yes
rewrite_contact=yes
force_rport=yes
language=fr
one_touch_recording=on
record_on_feature=apprecord
record_off_feature=apprecord
media_encryption=dtls
dtls_verify=no
dtls_cert_file=/etc/asterisk/keys/default.crt
dtls_private_key=/etc/asterisk/keys/default.key
dtls_setup=actpass
dtls_rekey=0

In order to configure webrtc on asterisk I have mainly used https://wiki.asterisk.org/wiki/display/AST/WebRTC+tutorial+using+SIPML5

Now, back to the virtual extension 1003.

In order for the GDS to be able to call 1004 (webrtc), it means that this extension must be reachable :wink:
However, when the tablet goes to sleep or when the view on the tablet is not the webrtc page, it is extremely likely that the webrtc extension will be UNreachable. So, if the GDS calls 1004, it won’t work as the extension is not here…

The “trick” here is the wake up the tablet, make sure the view with the webrtc card is shown and that the connection is available and then, call it.

The 1003 (virtual extension) dial plan is:

exten => 1003,1,System(curl http://IP_HASS:1880/endpoint/wakeup_webrtc)
exten => 1003,n,Wait(3)
exten => 1003,n,RetryDial(wait.wav,1,10,PJSIP/1004)
exten => 1003,n,Hangup()

The first line, calls a flow on HASS to wake up the webrtc extension.
The second is to make sure that potential old connections are killed during the wake up process.
The third tries to call 10 times the webrtc extension with 1 second interval. This gives some time for the wakeup to happen. It usually takes 2 to 3 seconds in my case. The 10 is to be on the safe side!

This is what you have to do on asterisk/freepbx.

Now, HA…

  1. Home assistant part
    2.a Follow @rdehuyss explanations from: https://github.com/rdehuyss/DoorDroid/wiki/DoorDroid-installation#homeassistant

You will not need the android app (not needed here) and setting up asterisk (it’s done…)

I have made very few changes to doordroid.js. Here is “my” version:
The buttons are using French. I have remove some features from the version above.
I have added the call to an input_boolean is HA.

class DoorDroidCard extends HTMLElement {

    constructor() {
        super();
        this.attachShadow({ mode: 'open' });
    }

    set hass(hass) {
        if(this.notYetInitialized()) {
            this.initJsSIPIfNecessary(hass);
            this.initCameraView(hass);
        } else if(this.cameraEntityHasNewAccessToken(hass)) {
            this.updateCameraView(hass);
        }
//        this.initDoorbellRinging(hass)
    }

    setConfig(config) {
        if (!config.camera_entity) {
            throw new Error('You need to define a camera entity');
        }
        if(!config.sip_settings) {
            throw new Error('You need to define the SIP settings');
        } else {
            if(!config.sip_settings.sip_wss_url) throw new Error('You need to define the SIP Secure Webservice url');
            if(!config.sip_settings.sip_server) throw new Error('You need to define the SIP Server (ip or hostname)');
            if(!config.sip_settings.sip_username) throw new Error('You need to define the SIP username');
            if(!config.sip_settings.sip_password) throw new Error('You need to define the SIP password');
        }
        this.config = config;
        const root = this.shadowRoot;
        if (root.lastChild) root.removeChild(root.lastChild);


        const card = document.createElement('ha-card');
        const content = document.createElement('div');
        const style = document.createElement('style');
        style.textContent = `
            ha-card {
                /* sample css */
            }
            .button {
                overflow: auto;
                padding: 16px;
                text-align: right;
            }
//            #cameraview img{
//                object-fit: cover;
//                height: 400px;
//           }
            mwc-button {
                margin-right: 16px;
            }
            `;
        content.innerHTML = `
        <div id='cameraview'>
            <p style="padding: 16px">Initializing SIP connection and webcam view</p>
            <audio id='audio-player'></audio>
        </div>
        <div class='button'>
            <mwc-button raised id='btn-open-door'>` + 'Ouvrir le portail' + `</mwc-button>
            <mwc-button style='display:none' raised id='btn-accept-call'>` + 'Accepter appel' + `</mwc-button>
            <mwc-button style='display:none' raised id='btn-reject-call'>` + 'Rejeter appel' + `</mwc-button>
            <mwc-button style='display:none' raised id='btn-end-call'>` + 'Terminer appel' + `</mwc-button>
        </div>
        `;
        card.appendChild(content);
        card.appendChild(style);
        root.appendChild(card);
    }

    // The height of your card. Home Assistant uses this to automatically
    // distribute all cards over the available columns.
    getCardSize() {
        return 1;
    }

    notYetInitialized() {
        return window.JsSIP && !this.sipPhone && this.config;
    }

    initJsSIPIfNecessary(hass) {
        console.log('Loading SIPPhone');

        let socket = new JsSIP.WebSocketInterface(this.config.sip_settings.sip_wss_url);
        let configuration = {
            sockets  : [ socket ],
            uri      : `sip:${this.config.sip_settings.sip_username}@${this.config.sip_settings.sip_server}`,
            password : this.config.sip_settings.sip_password
        };
        this.sipPhone = new JsSIP.UA(configuration);
        this.sipPhone.start();
        let droidCard = this;
        let openDoorBtn = droidCard.getElementById('btn-open-door');
        openDoorBtn.addEventListener('click', function(opendoor) {
            hass.callService('input_boolean', 'turn_on', { entity_id: 'input_boolean.open_portail' });
        });

        let callOptions = { mediaConstraints: { audio: true, video: false } };// only audio calls

        this.sipPhone.on("registered", () => console.log('SIPPhone registered with SIP Server'));

        this.sipPhone.on("newRTCSession", function(data){
            let session = data.session;
            if (session.direction === "incoming") {
                hass.callService('input_boolean', 'turn_on', { entity_id: 'input_boolean.gds_ringing' });
                let acceptCallBtn = droidCard.getElementById('btn-accept-call');
                let rejectCallBtn = droidCard.getElementById('btn-reject-call');
                let endCallBtn = droidCard.getElementById('btn-end-call');

                session.on("accepted", () => {
                    console.log('call accepted')
                    acceptCallBtn.style.display = 'none';
                    rejectCallBtn.style.display = 'none';
                    endCallBtn.style.display = 'inline-flex';
                });
                session.on("confirmed", () => console.log('call confirmed'));
                session.on("ended", () => {console.log('call ended'); droidCard.cleanup(hass)});
                session.on("failed", () =>{console.log('call failed'); droidCard.cleanup(hass)});
                session.on("peerconnection", () => {
                    session.connection.addEventListener("addstream", (e) => {
                        console.log('adding audiostream')
                        // set remote audio stream (to listen to remote audio)
                        // remoteAudio is <audio> element on page
                        const remoteAudio = document.createElement('audio');
                        //const remoteAudio = droidCard.getElementById('audio-player');
                        remoteAudio.srcObject = e.stream;
                        remoteAudio.play();
                    })
                });
                acceptCallBtn.addEventListener('click', () => {
                    session.answer(callOptions);
                    hass.callService('input_boolean', 'turn_off', { entity_id: 'input_boolean.gds_ringing' });
                });
                endCallBtn.addEventListener('click', () => session.terminate());
                rejectCallBtn.addEventListener('click', () => {
                      hass.callService('input_boolean', 'turn_off', { entity_id: 'input_boolean.gds_ringing' });
                      session.answer(callOptions);
                      setTimeout(() => {
                        session.terminate();
                      }, 1000);
                });

                acceptCallBtn.style.display = 'inline-flex';
                rejectCallBtn.style.display = 'inline-flex';
            }
        });
    }

    initCameraView(hass) {
        this.cameraViewerShownTimeout = window.setTimeout(() => this.isDoorPiNotShown() , 15000);
        const cameraView = this.getElementById('cameraview');
        const imgEl = document.createElement('img');
        const camera_entity = this.config.camera_entity;
        this.access_token = hass.states[camera_entity].attributes['access_token'];
        imgEl.src = `/api/camera_proxy_stream/${camera_entity}?token=${this.access_token}`;
        imgEl.style.width = '100%';
        while (cameraView.firstChild) {
            cameraView.removeChild(cameraView.firstChild);
        }
        cameraView.appendChild(imgEl);
        console.log('initialized camera view');
    }

    updateCameraView(hass) {
        const imgEl = this.shadowRoot.querySelector('#cameraview img');
        const camera_entity = this.config.camera_entity;
        this.access_token = hass.states[camera_entity].attributes['access_token'];
        imgEl.src = `/api/camera_proxy_stream/${camera_entity}?token=${this.access_token}`;
    }

    cameraEntityHasNewAccessToken(hass) {
        clearTimeout(this.cameraViewerShownTimeout);
        this.cameraViewerShownTimeout = window.setTimeout(() => this.isDoorPiNotShown() , 15000);

        if(!this.sipPhone) return false;
        const old_access_token = this.access_token;
        const new_access_token = hass.states[this.config.camera_entity].attributes['access_token'];

        return old_access_token !== new_access_token;
    }

    isDoorPiNotShown() {
        const imgEl = this.shadowRoot.querySelector('#cameraview img');
        if(!this.isVisible(imgEl)) {
            this.stopCameraStreaming();
        }
    }

    stopCameraStreaming() {
        console.log('Stopping camera stream...');
        const imgEl = this.shadowRoot.querySelector('#cameraview img');
        imgEl.src = '';
        this.access_token = undefined;
    }

    isVisible(el) {
        if (!el.offsetParent && el.offsetWidth === 0 && el.offsetHeight === 0) {
            return false;
        }
        return true;
    }

    cleanup(hass) {
        let acceptCallBtn = this.getElementById('btn-accept-call');
        let rejectCallBtn = this.getElementById('btn-reject-call');
        let endCallBtn = this.getElementById('btn-end-call');

        //acceptCallBtn remove eventlisteners and hide
        let clonedAcceptCallBtn = acceptCallBtn.cloneNode(true)
        clonedAcceptCallBtn.style.display = 'none';
        acceptCallBtn.parentNode.replaceChild(clonedAcceptCallBtn, acceptCallBtn);

        //rejectCallBtn remove eventlisteners and hide
        let clonedRejectCallBtn = rejectCallBtn.cloneNode(true)
        clonedRejectCallBtn.style.display = 'none';
        rejectCallBtn.parentNode.replaceChild(clonedRejectCallBtn, rejectCallBtn);

        //endCallBtn remove eventlisteners and hide
        let clonedEndCallBtn = endCallBtn.cloneNode(true)
        clonedEndCallBtn.style.display = 'none';
        endCallBtn.parentNode.replaceChild(clonedEndCallBtn, endCallBtn);

    }


    getElementById(id) {
        return this.shadowRoot.querySelector(`#${id}`);
    }
}

customElements.define('doordroid-card', DoorDroidCard);

Adapt the settings of the doordroid card to what you have done in 1.
Create a lovelace view such as:

title: Surveillance
icon: mdi:bell
panel: true
path: gds
cards:
      - camera_entity: camera.gds
        style: |
            ha-card {
              display: block;
              margin-left: auto;
              margin-right: auto;
              width: 88%;
            }
        sip_settings:
          sip_password: MySecureStuff
          sip_server: MYPBX
          sip_username: '1003'
          sip_wss_url: 'wss://MyPBX:8089/ws'
        type: 'custom:doordroid-card'
        title: DoorDroid

The camera.gds is the RTSP stream of the video of the door bell.

So, with this, normally, asterisk is ready, your lovelace/HA configuration is ready.

If you go the the lovelace/gds view and call from the GDS, you will have no ringing sound, but you should be able to pickup, talk and terminate the call. Almost there :slight_smile:

1 Like
  1. The node red part

3.a Waking up the tablet
I have found that the most efficient way to make sure that the webrtc is working on the tablet is to reload the lovelace page.
As I am using Fully Kiosk, there is an api for that:

WakeGDS is the endpoint called by the 1003 dialplan (see asterisk config). LoadGDS is an API call to reload the lovelace view with the doordroid card:

Then what to do when someone is called… Make some noise!

In my case, it is just playing sound on the tablet. You could do pretty much what you want. It is an automation after all!!

The last bit is to “open the door” when you click on “open the door”.

It is another automation on nodered…

Obviously, all of this need to be tailored to your setup. Another SIP doorbell, not the same PBX,…

Enjoy !!

GV

1 Like

Hi @greengolfer,
I am working on a similar project following some of your advice in this thread. At the moment, I am struggling with the FreePBX accepting SSL connections from the WebRTC Client. Could you provide some guidance on this? Here are a few questions that may help me:

  • Did you use a “Trusted” certificate or a “Self-signed” certificate for the websocket (wss://MyPBX:8089/ws’)? And if “Self-signed” did you get this to work with Google Chrome based WebRTC?
  • I believe FreePBX uses the “Asterisk Builtin mini-HTTP server” (Settings->Advanced Settings) for this websocket. The FreePBX default for this server uses keys/Certificate from /etc/asterisk/keys/integration/certificate.pem and webserver.key. Did you use this certificate/key, or did you instead use a self-signed or trusted certificate?
  • For the endpoint, you have the dtls setup to use /etc/asterisk/keys/default.crt and default.key. Are these the original default certificate/keys that came with FreePBX?

Thanks.

I am using a trusted certificate. I have my own domain. I have a wildcard certificate from cloudflare (free and lasts 15 years !!).
So, in my example MyPBX is the FQDN of my PBX. And not the IP address.

It is also the case in the lovelace view with webrtc. The wss:// is using the FQDN.

The same certificate is also uploaded in the PBX for HTTPS request.

Only the dtls in the extension uses the default cert that is available (I don’t remember creating those). I don’t think I have tried with my cloudflare cert…

GV

1 Like

Thanks, for your help :slight_smile: . I used my duckdns cert for now and this has solved my ssl problem.

I’m now struggling with my webRTC endpoint at FreePBX.
As a reference, I have setup an extension 1001 for my WebRTC.
I’m getting a log entry in FreePBX:
NOTICE[16660]: chan_sip.c:28939 handle_request_register: Registration from '<sip:[email protected]>' failed for 'MY_WAN_IP' - Wrong password

I have copy-pasted the password from FreePBX GUI (PJSIP Extension 1001->General->“Edit Extension”->“Secret” (copy)) into the HA Doordroid Lovelace (config sip_password: (paste)), so I don’t think that is the issue…

I noticed that the error is coming from chan_sip, whereas I setup the WebRTC in FreePBX using pjsip.
In fact I was not getting any output from pjsip logger, So I decided to turn on the sip debugger and I now get output for SIP messages from my WebRTC doordroid. This tells me pjsip is not the handler for the WebRTC.

May I ask, did you setup your WebRTC in FreePBX for chan_pjsip or chan_sip?
I will also mention I have pjsip setup (Settings->Asterisk SIP Settings) for both ws:0.0.0.0 and wss:0.0.0.0 set to “yes”…so puzzled about this.

Thanks,

Everything is using pjsip.
In my advance settings page, I have configured this:

This way, I am sure that pjsip is the only used.
In the “sip seetings” and the pjsip tab, I have only udp (0.0.0.0) and wss (0.0.0.0) set to yes.

You are almost there :slight_smile:

It took me a few weeks and “some” hours :frowning:

All the best,

GV

1 Like

That was it! Thanks! (Merci beaucoup!)
I was even successful getting a softPhone (Zoiper) to call the webrtc client. Awesome!
I’ve got some work to do still…In my project, I actually need to get the webrtc client to place the call to the doorbell…we’ll see how that goes. Again thanks!

1 Like

All good then (almost) !!
You will need to tweak the doordroid.js to add what is necessary to make outgoing calls.
I have used (when I was able to understand it) the documentation of jssip https://jssip.net/documentation/3.3.x/getting_started/
It was a bit cryptic for me though. Good luck.
I have also used https://www.pjsip.org/pjsua.htm in order to test my setup.
You could, for example, use pjsua to make (with a cli) a call to your doorbell and to your tablet and then have them talk (like automatic transfer). At some point I thought I had a use case for that, but no. However, it is doable.
If you are successful, you can share what you’ve done :slight_smile:
GV

Hi GV,
I am working on DoorPI. I am trying to add a button to the doorpi-card.js. I have no idea how to program in JS. I have been able to add the button `, but I am not sure how to link it with Hassio. Any assistance would be much appreciated.
Thanks
Mark

Unfortunately, my JS knowledge is also non existent. I just took the dordroid.js and by trial and error add what was needed in my case. I don’t really know how I can help.
Try to understand the code… is the only advice I can give you.
Sorry
GV

Hi GV thanks for your reply.
Are you new buttons using MQTT. I was hoping not to use MQTT, and was wondering if this is possible?
Thanks
Mark

Hi Mark,
The buttons are not using MQTT.

eg. this piece of code.
I have two input_booleans, one for ringing and one for opening the door. They are turn on and off within JS directly.
Only the subsequent automation part is using NR.

GV

Hi GV,
Thank you so much. I will now start leaning JS.

Mark

Hello. Can you make this components for HA

You mean as an “official” integration? That would very good indeed, but I am totally unable (total lack of knowledge here) to do it.
Sorry.

GV

I wonder if this setup can be used for a room-to-room intercom using android devices?

At the moment, the android tablet is not making calls. So, in order to have the two tablets being able to talk to each other, you have to be able to call the other one. For this, you will need to modify the doordroid card. Add a call button or something like that.
Obviously, each tablet will need to have its own extension.
Doable, probably, easy, maybe not. Especially if you are like me, with limited knowledge in javascript.

GV

As an FYI, I have modified the Doordroid card to be able to call out as well as accept a call. I use it as part of an ongoing project and can make it available later on.

5 Likes