Parsing/extracting information from emails for utility bills

Configuration
,
1

Hello HA community! As my name would suggest, I am a “noob” at home assistant. However I posted this question but before I could get help I figured out a way to extract what I needed so will post this for someone who ends up here from a search.

Goal: To extract a dollar value to be presented in home assistant to know what was last months bill for different services. For this to work, the email from your utility company must include the bill amount in the body and not in PDF or something else.

What I put in my Configuaration.yaml:

- platform: imap_email_content
  name: Bills Email
  server: imap.gmail.com
  port: 993
  username: [email address]
  password: [password]
  folder: Bills
  senders:
   - [senders email goes here]

This works to create a sensor for me

image
image2070×444 32.1 KB

The "bill2’ is my subject from the test email I sent to the location to see if it works and the body is the actual content.

{{ state_attr('sensor.bills_email','body')
    | regex_findall ("\$([0-9.]+)") }}

I used the developer tool and tested it. It spit back 29 which is the amount in my body as the picture shows. I used the above in a mark down card in lovelace to show me the amount.

After my first successful try. I created different folders for different utilities and forwarded those emails. If I figure out a way to add PDF’s someway, I will add that too. There is a post Parsing PDF email attachments - Share your Projects! / Custom Components - Home Assistant Community (home-assistant.io) doing exactly this with PDF but I could not get it to work.

I am sure there might have been a simpler or better way of doing this and if there is let me know. But for now, without any coding experience, I am going with this.

2 Likes
3

Self Solved. Please look at the main post.

4

That’s a decent solution for parsing email body, nice work. It’s the same regex strategy that the imap_email_attachment integration uses. This integration is languishing a bit, there is some work to be done to get it up to standards again, apologies.

5

No apologies needed. Thanks for sharing your work with PDF parsing. I do get some emails with PDF so looking to work on integrating your method for parsing PDF’s into my setup. Just trying to find a good long weekend to work on this. I am sure I didnt do something right the first time around.

6

my friend I have completely given up on HA for pulling information from imap bill emails. imap_email_content has a poor integration makes things very difficult it’s fine for pulling one type of email for one sensor. that’s about it. I ended being able to do everything through node red with a the email node, filter the emails, regex the values and send it all to my mqtt server, where HA would pick it up as a mqtt sensor. Nodered def has it’s own quirks but it intuitive with things like this.

7

Hi @Noob25930
I’m afraid your configuration.yaml example does not work as of now - either that or I am missing something from your explanation :confused: would you mind having a quick glance at it?
I have set up the IMAP integration, but if I try to put the code you posted on my configuration.yaml, I get the error:

Platform error 'sensor' from integration 'imap_email_content' - Integration 'imap_email_content' not found.

Thank you in advance!

8

You need to set up a sensor from the event:

10

@Holdestmade gave a good solution. It seems you dont have IMAP setup properly on your home assistant. The integration has been working solid for me so I assume there is a problem with the sensor setup. I will say that it took alot of learning regex to get to the point of extracting exactly what I wanted. Maybe try not using any regex and see if your email and body shows up in Home Assistant first. Please let me know if you still are having trouble. Id be happy to help.

1 Like
11

Sorry for the late reply but honestly I would like to learn more node red. any chance you can share the flow without your private information? I want to see if I can learn from it and adopt in my own setup. I have heard great things about node red.

12

It only took the whole weekend to finally have it set up :smiley: but it is working now - the regex was the least of the problems, rather configuring the sensor and the event capturing was the most annoying part.

That, and well the formatting of the number, for any future investigator, if the retrieved number has some spaces in between (for example “123 456.32”) and you have set up the unit of the sensor to a currency, for example EUR, you do need to remove the spaces, or it will just show “Unknown” - took me a bit to grasp.

I want to jump in also on the node red - I gave up on the regex approach and considered node red, but it straight down discouraged me even more when I got to it and saw I understood even less about it… @mrkricfalusi we’d love to get some insights about your setup!

13

Hey there!
Coming back to ask… my IMAP integration is marking all my emails as read automatically!! Any idea how I can fix it?

14

It is not normal for that to happen unless you are using the fetch and processing functions. However, we have no idea what you did… post any related configuration.

15

I agree with @Didgeridrew, mine does not exhibit the same behavior. Although I have created separate email address with filters and folders for specific emails to be forwarded and read by home assistant. I would advise for you to do the same as in the case of your home assistant being compromised, there is no way to access your main email address.