Extract email info to HA

1

I have 3 (separate) propane tanks each monitored with a Nee-Vo propane level monitor supplied by the gas company.

As much as I try, Nee-Vo won’t open their API.

So, all I have is a once daily email for each of the 3 tanks that includes the tank number, date/time, and percentage full (from which it’s easy to derive the number of gallons remaining because it also idenfities the tank size).

I was wondering if anyone had a way to extract the info from the emails, upload them into some sort of database or directly into HA?

Thank you!

This is the body of the emails that come in each day:

Hello Joseph xxxxxx,

This is a periodic report of your tank level.

Tank:
201xxxx1

Inventory:
37 % (89 / 240 US gallons )

Time:
Saturday, January 29, 2022 5:45 AM (Eastern Standard Time)

Product:
Propane

Serial Number:
201xxxx1

2

See

… from just recently.

3

That’s a great solution – Thank you!

I’ve been trying to get it to work. I got the imap connector set up, but I’m stuck on the | regex filter to extract the info needed.

Can someone help me?

The template code I have so far is:

template:
  - sensor:
      - name: "TankID"
        unit_of_measurement: ""
        state: >
          {{ state_attr('sensor.neevo','body')
          | regex_findall_index("20196761") }}
      - name: "neevo_subject"
        unit_of_measurement: ""
        state: >
          {{ state_attr('sensor.neevo','subject') }}

The body of the message contains this (3 different tanks, each with different tank numbers; one email per day for each tank with an inventory: amount showing the number of gallons remaining, and the time: the reading occurred:

> <!DOCTYPE html><html><head>=0A <title></title>=0A =0A</head= >=0A<body style=3D"background: #F7F7F7;-webkit-border-radius: 5px= ;-moz-border-radius: 5px;border-radius: 5px;font-family: Calibri,= Arial;font-size: 18px">=0A <table border=3D"0" width=3D"100%"= cellspacing=3D"0" cellpadding=3D"10">=0A <tbody>=0A = <tr>=0A <td align=3D"center" style=3D"font-f= amily: Calibri, Arial;font-size: 18px;color: #111111;padding-left= : 5px">=0A <table border=3D"0" width=3D"700" c= ellspacing=3D"0" cellpadding=3D"0" style=3D"text-align: left;">=0A= <tbody><tr>=0A = <td valign=3D"top" style=3D"font-family: Calibri, Arial;font-siz= e: 18px;color: #111111;padding-left: 5px;padding: 15px">=0A = <p style=3D"font-family: Calibri, Arial= ;font-size: 18px;color: #111111">Hello Joseph x. xxxxxxx,</p>=0A = <p style=3D"font-family: Calibri, A= rial;font-size: 18px;color: #111111">This is a periodic report of= your tank level.</p>=0A <p style=3D= "font-family: Calibri, Arial;font-size: 18px;color: #111111">=0A = <label style=3D"color: #222222= ;padding-right: 5px">=0A T= ank:=0A </label>=0A = <br>=0A = <span style=3D"font-family: Calibri, Arial;font-size: 18pt;col= or: #000000;font-weight: bold">=0A = 20527575=0A </span>=0A= </p>=0A=0A = <p style=3D"font-family: Calibri, Arial;font-size: 18px;= color: #111111">=0A <label sty= le=3D"color: #222222;padding-right: 5px">=0A = Inventory:=0A = </label>=0A <br>=0A = <span style=3D"font-family: Calibri, = Arial;font-size: 18pt;color: #000000;font-weight: bold">=0A = 78 %=0A = </span>=0A <span s= tyle=3D"font-family: Calibri, Arial;font-size: 18px">=0A = (94 / 120=0AUS gallons = )=0A = </span>=0A </p>=0A=0A = <p style=3D"font-family: Calibri, Arial;fon= t-size: 18px;color: #111111">=0A = <label style=3D"color: #222222;padding-right: 5px">=0A = Time:=0A = </label>=0A <br>=0A= <span style=3D"font-family: C= alibri, Arial;font-size: 18pt;color: #000000;font-weight: bold">=0A= Wednesday, September 1, 2= 021 9:16 PM=0A </span>=0A = <span style=3D"font-family: Calibr= i, Arial;font-size: 18px">=0A = (Eastern Daylight Time)=0A = </span>=0A </p>=0A=0A = <p style=3D"font-family: Calibri, Arial;font-= size: 18px;color: #111111">=0A = <label style=3D"color: #222222;padding-right: 5px">=0A = Product:=0A = </label>=0A <br>=0A= <span style=3D"font-family: C= alibri, Arial;font-size: 18pt;color: #000000;font-weight: bold">=0A= Propane=0A = </span>=0A </= p>=0A=0A <p style=3D"font-family: = Calibri, Arial;font-size: 18px;color: #111111">=0A = <label style=3D"color: #222222;padding-right= : 5px">=0A Serial Number:=0A= </label>=0A = <br>=0A <spa= n style=3D"font-family: Calibri, Arial;font-size: 18pt;color: #00= 0000;font-weight: bold">=0A = 20527575=0A </span>=0A = </p>=0A </td= >=0A </tr>=0A <tr>=0A= <td valign=3D"top" style=3D"font-fami= ly: Calibri, Arial;font-size: 18px;color: #111111;padding-left: 5= px;padding: 15px">=0A <p style=3D"= font-family: Calibri, Arial;font-size: 18px;color: #111111">=0A = <span style=3D"font-family: Cal= ibri, Arial;font-size: 18px"></span>=0A = </p>=0A </td>=0A = </tr>=0A <tr>=0A = <td valign=3D"top" style=3D"font-family: Calibri, Arial= ;font-size: 18px;color: #111111;padding-left: 5px;padding: 15px">= =0A <p style=3D"font-family: Calib= ri, Arial;font-size: 14px;color: #7694ab">=0A = <span style=3D"font-family: Calibri, Arial;font-s= ize: 14px;color: #7694ab"><b>Disclaimer</b></span>=0A = <br>=0A = <span style=3D"font-family: Calibri, Arial;font-size: 14px;colo= r: #7694ab">This notification has been sent to you by Otodata on = the behalf of your fuel supplier. Otodata does not validate the a= uthenticity of the email addresses provided by the fuel supplier.= This email contains proprietary information and may be confident= ial. If you are not the intended recipient of this email, you are= hereby notified that any dissemination, distribution or copying = of this message is strictly prohibited. If you received this mess= age in error, please delete it immediately.</span>=0A = </p>=0A <p sty= le=3D"font-family: Calibri, Arial;font-size: 14px;color: #7694ab"= >=0A <span style=3D"font-famil= y: Calibri, Arial;font-size: 14px;color: #7694ab">Please do not r= eply to this e-mail as Otodata and your fuel supplier will not re= ceive your message.</span>=0A </p>= =0A </td>=0A </= tr>=0A <tr>=0A = <td valign=3D"top" style=3D"font-family: Calibri, Arial;font-size= : 18px;color: #7694ab;padding-left: 5px;background-color: #344654= ;padding: 15px">=0A <p style=3D"fo= nt-family: Calibri, Arial;font-size: 18px;color: #7694ab;margin: = 0;line-height: 14px">=0A <span= style=3D"font-family: Calibri, Arial;font-size: 16px;color: #ed9= 715;font-weight: bold;font-style: italic">OTODATA Wireless Networ= k</span>=0A <br>=0A = <span style=3D"font-family: Calibri, Ari= al;font-size: 12px;color: #7694ab">9280 Boul. de l'Acadie, Montre= al (QC) H4N 3C5, Canada</span>=0A = </p>=0A </td>=0A = </tr>=0A </tbody></table>=0A = </td>=0A </tr>=0A <tr>=0A <t= d height=3D"30" style=3D"font-family: Calibri, Arial;font-size: 1= 8px;color: #111111;padding-left: 5px"></td>=0A </tr>=0A= </tbody></table>=0A=0A</body></html>

Thank you!

4

How familiar are you with regular expressions? I don’t think what you’re doing there makes any sense — what you’re doing there is looking for the first instance of the string 20196761 and returning that. There isn’t one in your mail, so that won’t work. Maybe that’s the tank serial number?

Regular expressions aren’t really a great tool for extracting data from HTML. And it’s made—— worse because your email body isn’t really in plain ASCII — it is in “quoted printable” form. That’s what all those random =0A and other nonsense is coming from. Any result you get here is going to be _very fragile. But … we don’t have a lot of options here, so let’s see what we can do.

Try taking this:

{% set body='<!DOCTYPE html><html><head>=0A <title></title>=0A =0A</head= >=0A<body style=3D"background: #F7F7F7;-webkit-border-radius: 5px= ;-moz-border-radius: 5px;border-radius: 5px;font-family: Calibri,= Arial;font-size: 18px">=0A <table border=3D"0" width=3D"100%"= cellspacing=3D"0" cellpadding=3D"10">=0A <tbody>=0A = <tr>=0A <td align=3D"center" style=3D"font-f= amily: Calibri, Arial;font-size: 18px;color: #111111;padding-left= : 5px">=0A <table border=3D"0" width=3D"700" c= ellspacing=3D"0" cellpadding=3D"0" style=3D"text-align: left;">=0A= <tbody><tr>=0A = <td valign=3D"top" style=3D"font-family: Calibri, Arial;font-siz= e: 18px;color: #111111;padding-left: 5px;padding: 15px">=0A = <p style=3D"font-family: Calibri, Arial= ;font-size: 18px;color: #111111">Hello Joseph x. xxxxxxx,</p>=0A = <p style=3D"font-family: Calibri, A= rial;font-size: 18px;color: #111111">This is a periodic report of= your tank level.</p>=0A <p style=3D= "font-family: Calibri, Arial;font-size: 18px;color: #111111">=0A = <label style=3D"color: #222222= ;padding-right: 5px">=0A T= ank:=0A </label>=0A = <br>=0A = <span style=3D"font-family: Calibri, Arial;font-size: 18pt;col= or: #000000;font-weight: bold">=0A = 20527575=0A </span>=0A= </p>=0A=0A = <p style=3D"font-family: Calibri, Arial;font-size: 18px;= color: #111111">=0A <label sty= le=3D"color: #222222;padding-right: 5px">=0A = Inventory:=0A = </label>=0A <br>=0A = <span style=3D"font-family: Calibri, = Arial;font-size: 18pt;color: #000000;font-weight: bold">=0A = 78 %=0A = </span>=0A <span s= tyle=3D"font-family: Calibri, Arial;font-size: 18px">=0A = (94 / 120=0AUS gallons = )=0A = </span>=0A </p>=0A=0A = <p style=3D"font-family: Calibri, Arial;fon= t-size: 18px;color: #111111">=0A = <label style=3D"color: #222222;padding-right: 5px">=0A = Time:=0A = </label>=0A <br>=0A= <span style=3D"font-family: C= alibri, Arial;font-size: 18pt;color: #000000;font-weight: bold">=0A= Wednesday, September 1, 2= 021 9:16 PM=0A </span>=0A = <span style=3D"font-family: Calibr= i, Arial;font-size: 18px">=0A = (Eastern Daylight Time)=0A = </span>=0A </p>=0A=0A = <p style=3D"font-family: Calibri, Arial;font-= size: 18px;color: #111111">=0A = <label style=3D"color: #222222;padding-right: 5px">=0A = Product:=0A = </label>=0A <br>=0A= <span style=3D"font-family: C= alibri, Arial;font-size: 18pt;color: #000000;font-weight: bold">=0A= Propane=0A = </span>=0A </= p>=0A=0A <p style=3D"font-family: = Calibri, Arial;font-size: 18px;color: #111111">=0A = <label style=3D"color: #222222;padding-right= : 5px">=0A Serial Number:=0A= </label>=0A = <br>=0A <spa= n style=3D"font-family: Calibri, Arial;font-size: 18pt;color: #00= 0000;font-weight: bold">=0A = 20527575=0A </span>=0A = </p>=0A </td= >=0A </tr>=0A <tr>=0A= <td valign=3D"top" style=3D"font-fami= ly: Calibri, Arial;font-size: 18px;color: #111111;padding-left: 5= px;padding: 15px">=0A <p style=3D"= font-family: Calibri, Arial;font-size: 18px;color: #111111">=0A = <span style=3D"font-family: Cal= ibri, Arial;font-size: 18px"></span>=0A = </p>=0A </td>=0A = </tr>=0A <tr>=0A = <td valign=3D"top" style=3D"font-family: Calibri, Arial= ;font-size: 18px;color: #111111;padding-left: 5px;padding: 15px">= =0A <p style=3D"font-family: Calib= ri, Arial;font-size: 14px;color: #7694ab">=0A = <span style=3D"font-family: Calibri, Arial;font-s= ize: 14px;color: #7694ab"><b>Disclaimer</b></span>=0A = <br>=0A = <span style=3D"font-family: Calibri, Arial;font-size: 14px;colo= r: #7694ab">This notification has been sent to you by Otodata on = the behalf of your fuel supplier. Otodata does not validate the a= uthenticity of the email addresses provided by the fuel supplier.= This email contains proprietary information and may be confident= ial. If you are not the intended recipient of this email, you are= hereby notified that any dissemination, distribution or copying = of this message is strictly prohibited. If you received this mess= age in error, please delete it immediately.</span>=0A = </p>=0A <p sty= le=3D"font-family: Calibri, Arial;font-size: 14px;color: #7694ab"= >=0A <span style=3D"font-famil= y: Calibri, Arial;font-size: 14px;color: #7694ab">Please do not r= eply to this e-mail as Otodata and your fuel supplier will not re= ceive your message.</span>=0A </p>= =0A </td>=0A </= tr>=0A <tr>=0A = <td valign=3D"top" style=3D"font-family: Calibri, Arial;font-size= : 18px;color: #7694ab;padding-left: 5px;background-color: #344654= ;padding: 15px">=0A <p style=3D"fo= nt-family: Calibri, Arial;font-size: 18px;color: #7694ab;margin: = 0;line-height: 14px">=0A <span= style=3D"font-family: Calibri, Arial;font-size: 16px;color: #ed9= 715;font-weight: bold;font-style: italic">OTODATA Wireless Networ= k</span>=0A <br>=0A = <span style=3D"font-family: Calibri, Ari= al;font-size: 12px;color: #7694ab">9280 Boul. de l\'Acadie, Montre= al (QC) H4N 3C5, Canada</span>=0A = </p>=0A </td>=0A = </tr>=0A </tbody></table>=0A = </td>=0A </tr>=0A <tr>=0A <t= d height=3D"30" style=3D"font-family: Calibri, Arial;font-size: 1= 8px;color: #111111;padding-left: 5px"></td>=0A </tr>=0A= </tbody></table>=0A=0A</body></html>' %}

{{ body | regex_findall_index("/ (\d+)?=0AUS gallons") }}

into the template editor in the Developer section of your HA isntallation. The first line sets “body” to the string you’ve given above, and the second one filters that.

You’ll need to learn some basic regex syntax to figure out what exactly should go there.

5

I get it.

When in Outlook, I view the message “source” the HTML is nicely formatted (i.e., no “=0A”). But, the imap connector that actually downloads the message from my server shows the messier “quoted printable” form.

It would be much easier if the original message (or at least the way it is downloaded) were in text form.

I’ll have to work on this some more.

Thank you for your help!

6

Even though it’s showing you the “source”, Outlook is probably actually decoding the message. It’s pretty easy to do that if you can run arbitrary software, but the filters in the templates are somewhat limited. One thing you might be able to do is bounce it through some other email processing service or system that can clean up the formatting.

7

I’ve been heading down a number of different rabbit holes with this.

I thought maybe set up an Outlook rules to forward all of these messages to a dedicated email address using a VBA script to change the format to plain text

https://www.howto-outlook.com/howto/replyinhtml.htm

But it seems like a clunky solution.

8

Hi, I am having problem regarding this: How I can get the full "html source" or "original message" of the email body?

Can someone help me?