Intruder Alert Notification with Pictures from MJPEG Stream

I just thought I’d share this automation I have been working on as it took me a while to figure out. Might be useful to someone…

Back Story

A while ago I setup a door sensor on my front door. This is useful for a lot of automation tasks (such as lighting when we get home) but I also setup an automation so that we would get a notification (via Telegram) if the door was opened whilst neither of us were at home (Locative Tracking).

The issue I found was that very occasionally the automation would misfire due to the door opening before the location tracking had had a chance to catch up. This wasn’t so often as to make the automation useless but it did mean that the notifications were essentially ignored as we would both just assume it was just a misfire (not brilliant if there was a genuine intruder).

To fix this I purchased and setup a Raspberry Pi Zero W to act as an IP Camera. I used the official case and camera and it’s running MotionEyeOS. The idea being that when the door was opened (with no one at home) it could take some snaps of whoever came in and send them to us to verify.

Problem

I found with MotionEyeOS I could get a much better frame rate using the Fast Network Camera option. This unfortunately disables a lot of the OS’s great features such as motion tracking but for my use case this wasn’t a problem. What was a problem was that it also disables the URL to grab still images and leaves only the MJPEG stream up. If I wanted to send photos via Telegram then I needed a solution.

After a lot of Googling and playing around I eventually came across this solution:

automation:
  - alias: frontdoor_snapshots
    trigger:
      platform: mqtt
      topic: automation
      payload: "frontdoor"
    condition:
      condition: and
      conditions:
        - condition: state
          entity_id: device_tracker.b57a733cd0fc44957a733cd0fc4493894
          state: 'not_home'
        - condition: state
          entity_id: device_tracker.97097be2b976ce28e8e442cd0fc449449
          state: 'not_home'
    action:
      - service: shell_command.take_snapshots
      - service: shell_command.rename_snapshots
      - service: notify.telegram
        data:
          message: "Frontdoor Opened!"
      - service: notify.telegram
        data:
          message: "Frontdoor Opened"
          data:
            photo:
              - file: /home/pi/snapshots/snapshot1.jpg
                caption: "Snapshot 1"
              - file: /home/pi/snapshots/snapshot2.jpg
                caption: "Snapshot 2"
              - file: /home/pi/snapshots/snapshot3.jpg
                caption: "Snapshot 3"

As you can see, the trigger is an MQTT message which is sent by the door sensor. The condition is that nether of us are at home. The actions are as follows:

The first fires a Shell Command which uses VLC to take 3 snapshots from the MJPEG stream:

shell_command:
  take_snapshots: 'rm -f /home/pi/snapshots/* | vlc -q -I dummy http://192.168.0.11:8081 --video-filter=scene --vout=dummy --scene-format=jpg --scene-ratio=1 --scene-prefix=snapshot --scene-path=/home/pi/snapshots --run-time 5 --scene-ratio=40 vlc://quit'

The important bits in this command are:

rm -f /home/pi/snapshots/*: This clears the snapshots directory of all files (all old snapshots).
http://192.168.0.11:8081: The URL to my MJPEG stream
–scene-path=/home/pi/snapshots: The location where the snapshots should be saved.
–run-time 5: Only run for 5 seconds.
–scene-ratio=40: Only take a snapshot every 40 frames.

It took some trial and error but I eventually found that 5 seconds covered the person from walking in to walking past the camera (as they enter the house) and the “every 40 frames” meant that 3 snapshots were taken over the course of the 5 seconds.

The next issue I had was that the filenames of the snapshots were difficult to determine as they included the frame number (which could be slightly different each time it was run). A simple solution to this was to simply rename every file in the snapshots directory using a known format (snapshot<n>.jpg):

shell_command:
  rename_snapshots: ls /home/pi/snapshots | cat -n | while read n f; do mv "/home/pi/snapshots/$f" "/home/pi/snapshots/snapshot$n.jpg"; done

Using this I can specify the 3 filenames up front in the automation:

...
photo:
  - file: /home/pi/snapshots/snapshot1.jpg
    caption: "Snapshot 1"
  - file: /home/pi/snapshots/snapshot2.jpg
    caption: "Snapshot 2"
  - file: /home/pi/snapshots/snapshot3.jpg
    caption: "Snapshot 3"
...

The result is as follows (Please Note: I have pixelated the photos for privacy, the actual quality is very good :grinning:).

13 Likes

Very nice!

I’m doing the same thing but a little different. I call a bash script:

#!/bin/bash

rm -f /home/homeassistant/.homeassistant/snapshot/deurbel/*.jpg

wget -q -O /home/homeassistant/.homeassistant/snapshot/deurbel/deurbel.jpg http://USER:[email protected]:80/image/jpeg.cgi
convert /home/homeassistant/.homeassistant/snapshot/deurbel/deurbel.jpg \
      -pointsize 14 -fill white -undercolor black -annotate +5+15  \
      $(date +"%d/%m/%Y-%H:%M:%S") \
/home/homeassistant/.homeassistant/snapshot/deurbel/deurbel.jpg

It takes only 1 picture but I can easy add “sleep 2” and let it take an other picture.

E.

5 Likes

I do something similar but add each frame into a animated gif

2 Likes

An animated GIF is a great shout. I might have to give this a go. Cheers.

I am new to Home Assistant. How do you go about adding each frame into an animated gif? Do you have an example setup within Home Assistant that I could implement?

Unfortunately, I do not have a HA component for animated gif from stills. I have a security camera NVR which take 4 cameras and records stills when motion is detected to a ftp site. This is a common feature with security camera’s. I then mashed together several python scripts, one to rename the files, one to remove similar images and finally the link above to add the resulting images together into a gif image that HA reads into its generic image camera component.

Im searching auch kind of solution but with foscam nd xmpp

1 Like

Wow i thought i had ha great plan to set up my camera but then i found this thred :slight_smile: , thanks to all!. My work in progress solution is

action:
- service: shell_command.takort
- service: notify.telegram
  data:
    title: 'Home Assistant'
    message: 'Photo'
    data:
      document:
         file: /home/homeassistant/axis/a.gif
         caption: "Snapshot 1"

the takort.sh look like this
#!/bin/bash

rm -f /home/homeassistant/axis/*.jpg

wget -q -O /home/homeassistant/axis/front1.jpg http://***:***@192.168.1.21/jpg/image.jpg
sleep 3
wget -q -O /home/homeassistant/axis/front2.jpg http://***:***@192.168.1.21/jpg/image.jpg
sleep 3
wget -q -O /home/homeassistant/axis/front3.jpg http://***:***@192.168.1.21/jpg/image.jpg
convert -loop 0 -delay 100 /home/homeassistant/axis/front1.jpg /home/homeassistant/axis/front2.jpg /home/homeassistant/axis/front3.jpg /home/homeassistant/axis/a.gif

Now i get a gif file as notification!

7 Likes

Is this only possible with telegram?

Just tested with pushbullet but getting error.

I’ve just set this up by using BlueIris which saves jpegs once a second to 9 files in a loop.

Only tested in dev, hopefully I’ll find out how well it’s working when the post man arrives tomorrow!

Code and config to follow…

configuration.yaml

homeassistant:

  whitelist_external_dirs:
    - /home/homeassistant/.homeassistant/includes/snapshots

shell_command:
  snapshot_cam3_gif: /home/homeassistant/.homeassistant/includes/shell_commands/gif_cam3.sh
  snapshot_cam3_get: /home/homeassistant/.homeassistant/includes/shell_commands/get_cctv_cam3.sh

get_cctv_cam3.sh (don’t forget to chmod +x)

This is taking the x9 looped BlueIris Jpeg files from my local BlueIris server and saving them to my HomeAssistant server.

If BlueIris is running on same server as Home Assistant you could save directly, which would probably work better, or you could mount a shared drive/directory on BlueIris server from Home Assistant server and then skip this step altogether (I may do this myself).

wget -r -np -nH -R index.html http://192.168.1.11/cam3/ -P /home/homeassistant/.homeassistant/includes/snapshots/

gif_cam3.sh (don’t forget to chmod +x)

This has a dependency for imagemagick (On debian “sudo apt-get install imagemagick”)

Thanks @zidnuss I stole this from above :slight_smile:

#!/bin/bash
convert -loop 0 -delay 100 /home/homeassistant/.homeassistant/includes/snapshots/cam3/Cam3_009.jpg  /home/homeassistant/.homeassistant/includes/snapshots/cam3/Cam3_008.jpg /home/homeassistant/.homeassistant/includes/snapshots/cam3/Cam3_007.jpg /home/homeassistant/.homeassistant/includes/snapshots/cam3/Cam3_006.jpg /home/homeassistant/.homeassistant/includes/snapshots/cam3/Cam3_005.jpg /home/homeassistant/.homeassistant/includes/snapshots/cam3/Cam3_004.jpg /home/homeassistant/.homeassistant/includes/snapshots/cam3/Cam3_003.jpg /home/homeassistant/.homeassistant/includes/snapshots/cam3/Cam3_002.jpg /home/homeassistant/.homeassistant/includes/snapshots/cam3/Cam3_001.jpg /home/homeassistant/.homeassistant/includes/snapshots/cam3/Cam3_000.jpg /home/homeassistant/.homeassistant/includes/snapshots/snapshot_cam3.gif

BlueIris Config
Right Click on Cam > Camera Properties > Post >
image

Automation Home Assistant Config:

Sends an animated GIF to telegram when my letterbox is opened, plan to add this to my door bell and outside motion sensor also.

- alias: Letterbox
  initial_state: 'on'
  trigger:
    platform: state
    entity_id: sensor.letterbox
    to: 'open'
  action:
    - service: shell_command.snapshot_cam3_get
    - service: shell_command.snapshot_cam3_gif
    - delay:
        seconds: 1
    - service: telegram_bot.send_document
      data_template:
        target: [!secret chat_id_telegram_mike, !secret chat_id_telegram_workphone]
        file: /home/homeassistant/.homeassistant/includes/snapshots/snapshot_cam3.gif
        caption: "You've got mail! Someone opened the letterbox"
4 Likes

So the GIFs are working perfectly but for some reason Telegram doesn’t seem to handle them well. It plays them once but that’s it!

During my testing last night it was working fine so not sure if this is an issue with the app.

Insted of .GIF use .mp4 much better quality and smaler size
convert -delay 100 /home/homeassistant/axis/front1.jpg /home/homeassistant/axis/front2.jpg /home/homeassistant/axis/front3.jpg /home/homeassistant/axis/front4.jpg /home/homeassistant/axis/a.mp4

2 Likes

Could you save a movie 10 sec pre and then like 30 sec, then send this to iOS app?

how do i do this in hassio?

Good idea! I’ll give it a go

million times better! thanks!

gone from average of 7MB to 400k!

Plays better in Telegram too! :slight_smile:

1 Like

is there anyway to change user and file permission of converted imagemagic mp4 file as it changes the file “snapshot_cam3.mp4” to root and telegram is not able to send it, I tried using shell file like this -
#!/bin/bash
convert -delay 100 /home/pi/outside/Outside_009.jpg /home/pi/outside/Outside_008.jpg /home/pi/outside/Outside_007.jpg /home/pi/outside/Outside_006.jpg /home/pi/outside/Outside_005.jpg /home/pi/outside/Outside_004.jpg /home/pi/outside/Outside_003.jpg /home/pi/outside/Outside_002.jpg /home/pi/outside/Outside_001.jpg /home/pi/outside/Outside_000.jpg /home/pi/outside/snapshot_outside.mp4
sleep 20
sudo chmod 755 /home/pi/outside/snapshot_outside.mp4
sleep 5
sudo chown pi:pi /home/pi/outside/snapshot_outside.mp4

If i try to run the file from ssh it works file and change permissions also but running shell command from HA just change permission of file to 600 and user root.

Change the ownership to your home assistant user and group. Mine is user: “homeassistant” for example

I tried that also but the new file which is converted always have ownership of root and permission 600.

What is the script you use? does it have “sudo” at the start?