Templates in command line sensor + Octopus Energy API

Installation Home Assistant OS
42

Hi Will

I’m a bit confused - your settings for input_number and automation are fine, and the command you have in the sensor must be working to some extent if you’re getting results returned in the right ballpark. Having results of 50 and 2 in my mind would indicate some sort of issue with timestamping. As per my previous posts, Octopus usually updates the previous day’s measurements to their API between 6-10am, so between midnight and the update time you should usually get 0 results because the command is requesting results between 0000 and 2359 the previous day, which aren’t yet available.

I presume you’re replacing the xxxxxxxx part with your API key authorisation code obtained with the instructions I posted in post #15 above? → Templates in command line sensor + Octopus Energy API - #15 by ajoyce
Either way if it’s authorising, then great!

I have two suggestions:

  1. try appending a timezone to the times in your command and see if this changes anything:
    curl -u "APIKey:" "https://api.octopus.energy/v1/electricity-meter-points/MPN/meters/SERIAL/consumption/?period_from={{ (as_timestamp(now()) - 86400) | timestamp_custom("%Y-%m-%d", True) }}T00:00:00+01:00&period_to={{ (as_timestamp(now()) - 86400) | timestamp_custom("%Y-%m-%d", True) }}T23:59:59+01:00"

  2. switch to using the bash script version as per post #39

Let me know how you get on!

43

Hi,

I have been wrestling with this and could never get it to correctly work utill i tried post #39, i now get my yesterday usage figures howver the current figures do not show, they are just the json count’s I am wondering if it is anything to do with when your API account was establised why one methood would work and the other does not.

44

Hi Mo

This will only ever show yesterday’s energy usage figures - as I explained in earlier posts, the data accessible by the API only updates with the previous day’s usage results at between 6am and 12am. There is no real-time, same-day tracking, unless you can figure out a way to get the data direct from the smart meter!

If you’ve managed to get yesterday’s usage figures in input_number.electricity, and a JSON count in sensor.electricity, then you’ve got it working.

45

Ok Cool thank you

46

Good news, everyone!

I’ve looked at all the posts above and decided the way I’ve made these sensors is a total mess. A sensor that requires a separate input_number AND automation AND script to run properly?! Rubbish. And a pain in the backside to set up.

So I’ve kicked myself into learning a bit more bash scripting and am pleased to present new scripts that remove the need for both the automation and the input_number.

Here’s what you need:

configuration.yaml:

sensor:
  - platform: command_line
    name: electricity
    unit_of_measurement: 'kWh'
    scan_interval: 600
    command: 'sh /config/bash_scripts/electricity.sh'

  - platform: command_line
    name: gas
    unit_of_measurement: 'm3'
    scan_interval: 600
    command: 'sh /config/bash_scripts/gas.sh'

config/bash_scripts/electricity.sh:

STARTDATE=`date -d @$(( $(date +"%s") - 86400)) +"%Y-%m-%dT00:00:00"`
ENDDATE=`date -d @$(( $(date +"%s") - 86400)) +"%Y-%m-%dT23:59:59"`
STARTDATEPREV=`date -d @$(( $(date +"%s") - 172800)) +"%Y-%m-%dT00:00:00"`
ENDDATEPREV=`date -d @$(( $(date +"%s") - 172800)) +"%Y-%m-%dT23:59:59"`
COUNT=`curl -H "Authorization: Basic [YOUR AUTH CODE AS PER POST #15]" "https://api.octopus.energy/v1/electricity-meter-points/[COPY FROM OCTOPUS API SITE]/meters/[COPY FROM OCTOPUS API SITE]/consumption/?period_from=$STARTDATE&period_to=$ENDDATE" | python -mjson.tool | grep -c "consumption"`

if [[ "$COUNT" == 48 ]]
then
curl -H "Authorization: Basic [YOUR AUTH CODE AS PER POST #15]" "https://api.octopus.energy/v1/electricity-meter-points/[COPY FROM OCTOPUS API SITE]/meters/[COPY FROM OCTOPUS API SITE]/consumption/?period_from=$STARTDATE&period_to=$ENDDATE" | jq '[.. | objects | .consumption] | add' | awk '{printf "%0.2f\n",$1}' 
else
curl -H "Authorization: Basic [YOUR AUTH CODE AS PER POST #15]" "https://api.octopus.energy/v1/electricity-meter-points/[COPY FROM OCTOPUS API SITE]/meters/[COPY FROM OCTOPUS API SITE]/consumption/?period_from=$STARTDATEPREV&period_to=$ENDDATEPREV" | jq '[.. | objects | .consumption] | add' | awk '{printf "%0.2f\n",$1}'  
fi

config/bash_scripts/gas.sh:

STARTDATE=`date -d @$(( $(date +"%s") - 86400)) +"%Y-%m-%dT00:00:00"`
ENDDATE=`date -d @$(( $(date +"%s") - 86400)) +"%Y-%m-%dT23:59:59"`
STARTDATEPREV=`date -d @$(( $(date +"%s") - 172800)) +"%Y-%m-%dT00:00:00"`
ENDDATEPREV=`date -d @$(( $(date +"%s") - 172800)) +"%Y-%m-%dT23:59:59"`
COUNT=`curl -H "Authorization: Basic [YOUR AUTH CODE AS PER POST #15]" "https://api.octopus.energy/v1/gas-meter-points/[COPY FROM OCTOPUS API SITE]/meters/[COPY FROM OCTOPUS API SITE]/consumption/?period_from=$STARTDATE&period_to=$ENDDATE" | python -mjson.tool | grep -c "consumption"`

if [[ "$COUNT" == 48 ]]
then
curl -H "Authorization: Basic [YOUR AUTH CODE AS PER POST #15]" "https://api.octopus.energy/v1/gas-meter-points/[COPY FROM OCTOPUS API SITE]/meters/[COPY FROM OCTOPUS API SITE]/consumption/?period_from=$STARTDATE&period_to=$ENDDATE" | jq '[.. | objects | .consumption] | add' | awk '{printf "%0.2f\n",$1/11.36}' 
else
curl -H "Authorization: Basic [YOUR AUTH CODE AS PER POST #15]" "https://api.octopus.energy/v1/gas-meter-points/[COPY FROM OCTOPUS API SITE]/meters/[COPY FROM OCTOPUS API SITE]/consumption/?period_from=$STARTDATEPREV&period_to=$ENDDATEPREV" | jq '[.. | objects | .consumption] | add' | awk '{printf "%0.2f\n",$1/11.36}'  
fi

This should leave you with sensor.gas and sensor.electricity which will update on their own as and when the previous day’s energy usage figures get updated. No need for automations or input_numbers!

Quite a few bits of code to replace in this - see post #15 and https://octopus.energy/dashboard/developer/ for all the snippets you need.

Hope this is useful and tidies things up a bit for people - let me know if anything doesn’t work for some reason and I’ll be happy to help clear things up. Good luck!

6 Likes
47

The code I mentioned in my last post started working a few days after posting. I guess the API was down?

I have moved over to your new example and it’s working ok using the bash script.
I can only test the electric at the moment as the Gas meter didn’t get switched over correctly.

Now I just need to decide on how to display the information and record it correctly.

Will also look if there is an API for the outgoingOctopus once I get that setup.

Edit:

I have just performed

curl -u "api_key" "https://api.octopus.energy/v1/products/"

I got back 13 products the one of interest is

{"code":"OUTGOING-FIX-12M-19-05-13","direction":"EXPORT","full_name":"Outgoing Octopus 12M Fixed May 2019","display_name":"Outgoing Octopus 12M Fixed","description":"Outgoing Octopus flat rate pays you for all exported energy at a flat rate.","is_variable":false,"is_green":true,"is_tracker":false,"is_prepay":false,"is_business":false,"is_restricted":false,"term":12,"available_from":"2019-05-16T00:00:00+01:00","available_to":null,"links":[{"href":"https://api.octopus.energy/v1/products/OUTGOING-FIX-12M-19-05-13/","method":"GET","rel":"self"}],"brand":"OCTOPUS_ENERGY"},

So it looks like the API knows about the exporting outgoingOctopus

There is no mention of exporting in the API docs

I just tried

curl -u "api_key" "https://api.octopus.energy/v1/electricity-meter-points/MPAN/meters/Serial/export/"

But was greeted with {"detail": "Not found."} so I will send Octopus an email to see if they can shed any light.

48

Hi Guys,

Newbie to HA here, so please be patient.

I have tried to connect my api to my HA but I didn’t had any success. Followed and tried different examples here from previous posts but got nothing in HA.

I’m posting bellow what I have added to my files, so if anyone can help me I would really be very happy.

configuration.yaml
sensor:

  • platform: command_line
    name: electricity
    unit_of_measurement: ‘kWh’
    scan_interval: 600
    command: ‘sh /config/electricity.sh’

electricity.sh

STARTDATE=date -d @$(( $(date +"%s") - 86400)) +"%Y-%m-%dT00:00:00"
ENDDATE=date -d @$(( $(date +"%s") - 86400)) +"%Y-%m-%dT23:59:59"
STARTDATEPREV=date -d @$(( $(date +"%s") - 172800)) +"%Y-%m-%dT00:00:00"
ENDDATEPREV=date -d @$(( $(date +"%s") - 172800)) +"%Y-%m-%dT23:59:59"
COUNT=curl -H "Authorization: Basic my_API" "https://api.octopus.energy/v1/electricity-meter-points/yyyyyyyyy/meters/xxxxxxx/consumption/?period_from=$STARTDATE&period_to=$ENDDATE" | python -mjson.tool | grep -c "consumption"

if [[ “$COUNT” == 48 ]]
then
curl -H “Authorization: Basic my_API” “https://api.octopus.energy/v1/electricity-meter-points/yyyyyyyyy/meters/xxxxxxx/consumption/?period_from=$STARTDATE&period_to=$ENDDATE” | jq ‘[… | objects | .consumption] | add’ | awk ‘{printf “%0.2f\n”,$1}’
else
curl -H “Authorization: Basic my_API” “https://api.octopus.energy/v1/electricity-meter-points/yyyyyyyyy/meters/xxxxxxx/consumption/?period_from=$STARTDATEPREV&period_to=$ENDDATEPREV” | jq ‘[… | objects | .consumption] | add’ | awk ‘{printf “%0.2f\n”,$1}’
fi---------------------------------------------------------------------------------------------

I have no idea what am I doing wrong.

EDIT:
Note: I have tested the api with my key on “Talend API tester” in Chrome and I can get all values, from the consumed energy and from the tariff.

Thanks in advance
R

49

Hi @R_K

I can try to help but can you start by re-posting using the correct code formatting for all your code? It helps a great deal as formatting is hugely important in YAML and bash scripts. Thanks!

50

Hi @ajoyce,

I have done some progress, a coder friend helped me explaining that all the code should be correct but my api_key needed to be encoded in Base64 and to be honest, didn’t understand that on one of your initial posts… I would never had guest that by mysef.

I can get a value now on homeassistant. Now just need to tinker and create the graphs.

Best Regards,
R

1 Like
51

Thanks for this, Electricity now showing (just need them to install a Smart Gas Meter now).

52

I’m struggling with this but not sure if it’s because I’m a new customer.

How long did it take for your readings to come through? My account isn’t reporting half hour readings yet so I’m hoping it’s that…

53

If you don’t see the readings on the Octopus website this won’t work.
I had a problem with the SMART Gas meter and it took them a couple of months to get it working!

1 Like
54

Does the Gas Meter script work ok for everyone else?

I have copied the config from post 46

But my Gas reading has never updated. I know the api key is correct as my Electric one always gets updated. I can manually trigger an api call in the shell fine.

You can see my bash_scripts here homeassistant-config/bash_scripts at master · wills106/homeassistant-config · GitHub and my sensors here homeassistant-config/sensors.yaml at 60c0da3d33d1376bfd290da986836ed8c6038d25 · wills106/homeassistant-config · GitHub
Unless I am missing something obvious?

55

Hello! Great work on this , got the bash scripts working for gas and electric, just need octopus to sort themselves out now as at the moment my energy use is two days behind instead of one

How are you displaying the data in homeassistant ?

I’m currently using custom mini graph card, but seeing as the data is always a day behind, the dates shown on the graph is always wrong. Anyone got around this?

56

Hi everyone

I’ve changed the way it works AGAIN… mainly because previous ways were timing out and causing my instance to get memory creep. No idea why.

New version requires a REST sensor and a template sensor for each meter.

REST examples:

sensor:
  - platform: rest
    name: Electricity Data
    resource_template: >
      {% set starttime = (as_timestamp(now()) - (86400)) | timestamp_custom('%Y-%m-%d', True) + 'T00:00:00' %}
      {% set endtime = (as_timestamp(now()) - (86400)) | timestamp_custom('%Y-%m-%d', True) + 'T23:59:59' %}
      https://api.octopus.energy/v1/electricity-meter-points/YOUR END POINT/meters/YOUR METER/consumption/?period_from={{starttime}}&period_to={{endtime}}
    headers:
      Authorization: Basic YOUR AUTH
    value_template: "{{ value_json.count }}"
    scan_interval: 7200
    json_attributes:
      - results
      
  - platform: rest
    name: Gas Data
    resource_template: >
      {% set starttime = (as_timestamp(now()) - (86400)) | timestamp_custom('%Y-%m-%d', True) + 'T00:00:00' %}
      {% set endtime = (as_timestamp(now()) - (86400)) | timestamp_custom('%Y-%m-%d', True) + 'T23:59:59' %}
      https://api.octopus.energy/v1/gas-meter-points/YOUR END POINT/meters/YOUR METER/consumption/?period_from={{starttime}}&period_to={{endtime}}
    headers:
      Authorization: Basic YOUR AUTH
    value_template: "{{ value_json.count }}"
    scan_interval: 7200
    json_attributes:
      - results

Then the template sensors are as follows:

sensor:
  - platform: template
      electricity:
        unit_of_measurement: 'kWh'
        value_template: >
          {% if states('sensor.electricity_data') == "48" %}
          {{ (state_attr("sensor.electricity_data", "results") | map(attribute="consumption") | sum ) | round(2) }}
          {% else %}
          {{ states('sensor.electricity') }}
          {% endif %}
      gas:
        unit_of_measurement: 'm3'
        value_template: >
          {% if states('sensor.gas_data') == "48" %}
          {{ (state_attr("sensor.gas_data", "results") | map(attribute="consumption") | sum /11.36 ) | round(2) }}
          {% else %}
          {{ states('sensor.gas') }}
          {% endif %}

@SJHarrison1992 and others - Octopus seems to be dropping the ball somewhat at the moment in terms of consistent meter reading updates being broadcast to the API. If you log in to your account and then scroll down and click ‘View Smart Meter Consumption’ you can see the latest data that Octopus has pushed to the API for your account. For me, it’s currently lagging about 2 days behind whereas it used to be only a day behind. I’m going to push them about this and see if they will start pushing numbers live. In the meantime, if you want the 24hr totals for 2/3/4/x days ago, edit the number 86400 in the REST sensors and replace it with a multiple of 86400 depending on how many days ago you want it to read (i.e. 2 days ago - change to 172800, 3 days ago - change to 259200).

5 Likes
57

I’ve just tried to implement this and it’s not working - both the data sensors are empty. Am I missing something? I checked the URLs that are being generated and they are returning data.

Any help is appreciated!

58

I had to base64 encode the API key in the header

1 Like
59

Hi all,

@bleep-io thanks for the tip re: base64 encoding!

As well as that, I also had to add the basic prefix to my Authorization header when using the config provided by @ajoyce above.

i.e.

  headers:
    Authorization: "basic <base_64_encoded_key>"

Just in case others land here.

2 Likes
60

Thanks for your efforts with this. Has something changed. I’m only getting 47 results. 23:00-23:30 is the last one in the result set. I dont see 23:00 to 00:00.

Anyone else having this issue?

1 Like
61

Hi Guys,

NOOB here just trying to get started with HA and wanted to get my Octopus meter readings on display. I followed the instructions but only get an "Entity is non numeric"wrapper on the overview screen of HA. Found your post here about the base64 but couldn’t figure it out. Please could you expand further how I base64 the Octopus API key? I followed the original instructions and got the “Authorize: Basic xxxxxxxxxxxxxxxxxxxxxxx” output from the Talend website.

`

P.S. do I copy the entire Octopus API key including the first part "sk_live_ " or only the alpha numeric after the underscore?

`

Thanx and regards
Chris.