Decode email body from imap email content sensor

Hi,

I think this is what I have been looking for… I too have a dvr system that sends emails as an alert for motion, while the emails are in plain text I still need a way of pulling the camera number that detected the motion from the email body.

I’m currently using a binary sensor to detect ftp uploads on motion detection (using inotify), its a little slow and there is no way to distinguish which camera detected motion.

I have had some partial success with using the template editor but its a little beyond me can you possibly give me some pointers?

ta
Scott

Sure, could you take a picture of an email and post it? Are you familiar with Python? Appdaemon coding? What is the model of your NVR, mine is in the LNR6100 Series.

hi,

I am not familiar with python coding unfortunately and I’ve been struggling to install Appdaemon for the last two days (for use with hadashboard).

The DVR is model Sansco 1080N 4 channel.

email image attached… the DVR sends one at the start of the motion “DetectStart” and one at the end “DetectEnd”.

I use Hassbian currently, because I need to use inotifywait to watch for ftp uploads for motion, up until the recent switch I was using hass.io, which is a lot simpler for a non python guy like me but you can’t use scripts with it.

thanks
Scott

Let me know when you have appdaemon working. I have created some python code that will find the camera number where motion was detected. We can then put the pieces together to get you up and running. Also, rather than a picture of your email, cut and paste the body in this thread so I can test with actual data.
With dummied data (email_body below), I created the following code:

#!/usr/bin/env python
import appdaemon.plugins.hass.hassapi as hass
import base64
import string

class pavey(hass.Hass):
    def initialize(self):
        email_body = "Alarm event: MotionDetectEnd  Alarm Begin Time: 2018-06-21 13:24:34 Alarm input channel No: CAM02(2)"
        y = email_body.find('DetectEnd')
        if y == -1:
            self.log("DetectEnd not found")
        else:
            z =  email_body.split('(')
            camera_index = z[1]
            camera_where_motion_detected = camera_index[0]
            self.log(camera_where_motion_detected)

wonderful stuff and much appreciated thank you, I’m struggling on with Appdaemon, permission and ssl issues are hampering any progress and I may go back to hass.io to install Appdaemon assuming your script will work under it?

Emails as follows…

Alarm event: Motion DetectStart
Alarm Begin Time: 2018-06-21 19:45:13
Alarm input channel No.: BACK GARDEN(1)
Alarm device name: LocalHost
Sender IP address: 192.168.1.197

Alarm event: Motion DetectEnd
Alarm Begin Time: 2018-06-21 19:45:15
Alarm input channel No.: BACK GARDEN(1)
Alarm device name: LocalHost
Sender IP address: 192.168.1.197

Alarm event: Motion DetectStart
Alarm Begin Time: 2018-06-21 19:46:39
Alarm input channel No.: FRONT GARDEN(2)
Alarm device name: LocalHost
Sender IP address: 192.168.1.197

Alarm event: Motion DetectEnd
Alarm Begin Time: 2018-06-21 19:46:41
Alarm input channel No.: FRONT GARDEN(2)
Alarm device name: LocalHost
Sender IP address: 192.168.1.197

these are the sets of email that are received for the two cameras I have connected.

ta
Scott

My recommendation is to NOT install SSL at this point. I could never get it to work with dashboard. I found TOR to be a better option for accessing HA and Dashboard away from home.

I will have a look at tor, I’m currently setting up Hass.io, but feel tor may not be compatible with that, although I do have an OpenWrt router which may be an option if need be.

The reason I used ssl was for the home assistant iOS app as the household uses it for location tracking.

Edit… just seen tor in the Hassio addons

Oops, I was using ssl to be able to connect to my PI over the internet. I am not familiar with the requirement you just mentioned regarding IOS, so my comment about using tor may not apply in your circumstances.

my ISP uses a dynamic IP address for my residential broadband, so using duckdns with let’s encrypt means I only ever needed the duckdns address as the IP address assigned by my isp may change after a reboot of the router or what not, if I can use duckdns on the router instead of on the hassio machine it should help with the ssl certificate problems… unless anyone knows different?

seperately I have appdemon up and running on my new hassio install.

ta
Scott

Great, let me know what you need.

I know very little about Appdaemon, im assuming a lot, but using your code I have created ‘emailparse.py’ inside the appdaemon/apps directory, added the reference in apps.yaml…

 hello_world:
   module: hello
   class: HelloWorld
 email_parse:
   module: emailparse
   class: pavey

thats pretty much how far its got, until i’ve done some reading I haven’t a clue how to call the app let alone include it within hassio to parse the email and change a binary_sensors value!

I don’t like to be spoon fed and the information is out there somewhere, so let me go away and see what I can work out and I’ll get back to you when / if I get stumped.

thanks so far!

Scott

After spending the day playing with your code, I’ve modified it as below…

import appdaemon.plugins.hass.hassapi as hass
import base64
import string

class pavey(hass.Hass):
    def initialize(self):
        email_body = "Alarm event: MotionDetectStart (1)"
        y = email_body.find('DetectStart')
        z =  email_body.split('(')
        camera_index = z[1]
        if y == -1:
            self.log("DetectStart not found")
            self.set_state("sensor.motion", state = "clear", attributes = {"friendly_name": "Motion"})
        else:
            camera_where_motion_detected = camera_index[0]
            self.log(camera_where_motion_detected)
            self.set_state("sensor.motion", state = (camera_where_motion_detected), attributes = {"friendly_name": "Motion Detected"}

The state changes in hassio front end as expected in relation to the values I change in the email_body line, I realise I can run the required actions (light/turn_on) directly within the code, but I had ideas of using the sensor to run automations.

Of course I’ve not found a solution for the input from the email, I did have a stab at it using get_state but wasn’t successful, I’d appreciate a pointer on that one, the email arrives via the gmail_imap sensor if that matters.

thanks
Scott

Scott,

I like your approach, i.e., not being spoonfed, you learn more and it sticks that way. Let me share one of the basic things I learned working with a veteran. When you display code in this forum, start it and end it with three “tic” marks, see keyboard.
![image|690x232]
(upload://ndOykg7bZ5MjBeAxcyCTVyTc9Rn.jpg)
To illustrate, I will repeat your code here:

#!/usr/bin/env python
import appdaemon.plugins.hass.hassapi as hass
import base64
import string

class pavey(hass.Hass):
def initialize(self):
email_body = “Alarm event: MotionDetectStart (1)”
y = email_body.find(‘DetectStart’)
z = email_body.split(’(’)
camera_index = z[1]
if y == -1:
self.log(“DetectStart not found”)
self.set_state(“sensor.motion”, state = “clear”, attributes = {“friendly_name”: “Motion”})
else:
camera_where_motion_detected = camera_index[0]
self.log(camera_where_motion_detected)
self.set_state(“sensor.motion”, state = (camera_where_motion_detected), attributes = {“friendly_name”: “Motion Detected”}

This approach makes your code more readable.

I realise I can run the required actions (light/turn_on) directly within the code, but I had ideas of using the sensor to run automations.

When you code the state change of your imap sensor, the automation takes place. In my NVR system, I am able to change the subject of the email, so I make the “subject” “Alert”. In the IMAP sensor, the “subject” is the state value, so in the following line of code:

        self.listen_state(self.motion_detected,"sensor.backyard_motion", new="Alert")

the “listen_state” is looking at the subject line to see if it says “Alert”. If so, it triggers the motion detect routine. In the motion detect routine, the email body is parsed to determine what camera detected the motion. I don’t think you need the following code:

self.set_state(“sensor.motion”, state = (camera_where_motion_detected), attributes = {“friendly_name”: “Motion Detected”}

What do you want to happen when motion is detected? In my application I turn on a light near the motion and keep it on for 5 minutes as shown in this code where I have 3 possible lights to turn on, depending on where the motion occurs:

            if detected_by_camera == "3": 
                self.set_state("sensor.backyard_motion", new = "off")
                self.turn_on("light.linear_lb60z1_dimmable_led_light_bulb_level", brightness = "120")
                self.cancel_timer(self.camera_3_handle)                                
                self.camera_3_handle = self.run_in(self.timer_handler, 300, myentity = "light.linear_lb60z1_dimmable_led_light_bulb_level")
                self.log(self.get_state("light.linear_lb60z1_dimmable_led_light_bulb_level", attribute='friendly_name') + ", turned on due to motion - line 30")
            # courtyard light                
            elif detected_by_camera == "2": 
                if self.now_is_between("sunset + 02:00:00", "sunrise"):
                #if self.now_is_between("sunrise", "sunset"):
                    self.set_state("sensor.backyard_motion", new = "off")
                    self.turn_on("light.linear_lb60z1_dimmable_led_light_bulb_level_4", brightness = "120")
                    self.cancel_timer(self.camera_2_handle) 
                    self.camera_2_handle = self.run_in(self.timer_handler, 300, myentity = "llight.linear_lb60z1_dimmable_led_light_bulb_level_4")
                    self.log(self.get_state("light.linear_lb60z1_dimmable_led_light_bulb_level_4", attribute='friendly_name') + ", turned on due to motion - line 39")                
            # middle garage                
            elif detected_by_camera == "1":
                self.set_state("sensor.backyard_motion", new = "off")                
                self.turn_on("light.unknown_node_21_level", brightness = "120")
                self.cancel_timer(self.camera_1_handle)                 
                self.camera_1_handle = self.run_in(self.timer_handler, 300, myentity = "light.unknown_node_21_level")
                self.log(self.get_state("light.unknown_node_21_level", attribute='friendly_name') + ", turned on due to motion - line 48")                
            else: self.log("Old email dated "+email_date+" ignored")
            self.set_state("sensor.backyard_motion", new = "off")
            #self.notify("Motion detected from camera " + detected_by_camera)
            detected_by_camera = ''
#        else: self.log("Sun is not down")
  
    def timer_handler(self, kwargs):
        self.turn_off(kwargs['myentity'])
        self.log(self.get_state(kwargs['myentity'], attribute='friendly_name') + ", turned on due to motion, is now off")
        self.notify(self.get_state(kwargs['myentity'], attribute='friendly_name') + ", turned on due to motion, is now off") 

One other subtle point, when Home Assistant is restarted, the IMAP sensor rereads all the emails from the beginning of your inbox, at 30 second intervals, and will go through your detection routine and can send false information to you. There is reference to a routine on this forum that will delete all old emails if Home Assistant is restarted (I thought I had it bookmarked, but I don’t). In my case, I decided to manually delete the emails each day as I scan through to see what happened the night before, and in the interim I installed this code to ignore old email messages:

            now = datetime.now()
            now_string = str(now.hour)+':'+str(now.minute) +':'+str(now.second)
            message_string = email_date[17:25]  #Mon, 18 Jun 2018 15:21:43
            FMT = '%H:%M:%S'
            tdelta = datetime.strptime(now_string, FMT) - datetime.strptime(message_string, FMT)
            self.log(str(tdelta.total_seconds()))   
            if tdelta.total_seconds() > 600:
                detected_by_camera = "" #reading old email messages, bypass motion processing
            elif tdelta.total_seconds() < -300:  #There is a difference in time between NVR and Home Assistant
                detected_by_camera = "" #reading old email messages, bypass motion  processing 

I hope this helps!

this is what I ended up with, ill look at your last post with great interest as I can see you have made allowances for old messages.

import appdaemon.plugins.hass.hassapi as hass
import base64
import string

class pavey(hass.Hass):
    def initialize(self):
        x = self.get_state("sensor.alarmnotficationsgmailcom", attribute = "body")
        email_body = (x)
        y = email_body.find('DetectStart')
        z =  email_body.split('(')
        camera_index = z[1]
        if y == -1:
            self.log("DetectStart not found")
            self.set_state("sensor.motion", state = "clear", attributes = {"friendly_name": "Motion"})
            self.log(x)
        else:
            camera_where_motion_detected = camera_index[0]
            self.log(camera_where_motion_detected)
            self.set_state("sensor.motion", state = (camera_where_motion_detected), attributes = {"friendly_name": "Motion Detected"})
            self.log(x)```

What does your imap configuration look like?

Here is mine:

  - platform: imap_email_content
    name: 'Backyard Motion'
    server: imap.gmail.com
    port: 993
    username: xxxxxxxxxxx
    password: !secret gmail_password
    senders:
      - [email protected]

I show this because the configuration creates the friendly name so you don’t have to set it in code. I am wondering why you are not listening to the state change of the sensor…

self.listen_state(self.motion_detected,“sensor.backyard_motion”, new=“Alert”)

just re-reading your post, my apologies to you and the other forum users, I looked for the way to format the code but I couldn’t find it using the menu bar at the top of the post window.

I can certainly change the emails subject, the same as you I would like lights to come on to reflect the location at which the motion occurred.

My laptop battery just died, so will have to resume this later again thanks for your help thus far.

Scott

Im not sure I can listen for the sensor state using the email title as both the start motion and end motion emails contain the same title and I only want to listen for the start, although there is probably a way to parse the email body afterwards to check for end and ignore it.

ive amended my imap config to use a friendly name.

thanks
Scott

The example I provided parses the email body and eliminates the ‘end’ portion.

i did it much easier, just 1 line of code change to decode the body text :slight_smile:

see here, my reply to that thread

now in my automations, i can look for a keyword