Templates in command line sensor + Octopus Energy API

Installation Home Assistant OS
10

This is just weird! The log above suggests that everything is parsing correctly, yet it doesn’t yield the same result… I’ll try again with the non-template version and see what the log spits out and compare it.

Ok, progress. When it works with the non-template URL, the log shows:

2019-02-01 21:06:37 INFO (SyncWorker_16) [homeassistant.components.sensor.command_line] Running command: curl -H "Authorization: Basic xxxxxxxxxxxx" "https://api.octopus.energy/v1/electricity-meter-points/yyyyyyyyyyyy/meters/18P5016779/consumption/?period_from=2019-01-31T00:00:00"
2019-02-01 21:07:40 INFO (SyncWorker_2) [homeassistant.components.sensor.command_line] Running command: curl -H "Authorization: Basic xxxxxxxxxxxx" "https://api.octopus.energy/v1/electricity-meter-points/yyyyyyyyyyyy/meters/18P5016779/consumption/?period_from=2019-01-31T00:00:00"

So it looks like when there’s a template in there, it excludes the " from around the header and url, and therefore fails, but without the template, the " are preserved and it works

11

FIXED IT!

For anyone who has the same issue in future:

  - platform: command_line
    name: electricity
    value_template: '{{ value_json.count }}'
    command: >-
      curl -H '"'Authorization: Basic xxxxxxxxxxxx'"' '"'https://api.octopus.energy/v1/electricity-meter-points/yyyyyyyyyyyy/meters/18P5016779/consumption/?period_from={{ (as_timestamp(now()) - (24*3600)) | timestamp_custom("%Y-%m-%d", True) }}T00:00:00'"'

The solution was a) to put it in a new line and b) to surround double quotes with single quotes to protect them being stripped out by Jinja2 parsing (like this - ’ " ’ )

Thanks for steering me towards the solution everyone!

1 Like
12

Glad to hear you resolved the problem!

Not that it matters anymore but for the solution I had proposed, this is what I needed to do to make it work:

  • Specify the full path to the script file. In my case it was:
command: "/opt/homeassistant/config/test.sh"
  • Ensure the user that owns the Home Assistant process (in my case the user’s name is homeassistant) has execution permission for the script file.

Error messages “Command failed: test.sh” followed by “test.sh: not found” mean the specified path was wrong.

Error messages "“Command failed: test.sh” followed by “Permission denied” means the process did not have permission to access the script file.

13

Ah, I thought I might have screwed up the path - whoops! Thanks anyway!

14

Any chance you could post the working setup? I’m a relatively new Octopus customer and would like to see the energy usage in HA.

15

Sure:

  - platform: command_line
    name: [gas or electricity]
    value_template: '{{ value_json.count }}'
    json_attributes: 
      - results
    command: >-
      curl -H '"'Authorization: Basic [encoded API key here]'"' '"'https://api.octopus.energy/v1/[insert 'gas' or 'electricity' here]-meter-points/[mpan or mprn]/meters/[meter serial number here]/consumption/?period_from={{ (as_timestamp(now()) - (24*3600)) | timestamp_custom("%Y-%m-%d", True) }}T00:00:00'"'

Note that for authorization you will need the encoded API key - easiest way to do this is to get your API key, go to https://client.restlet.com, click ‘set an authorization’, copy the API key into the username box, and click ‘set’ - you should then see the header formed with 'Basic + [encoded API key].

This should return a state of ‘48’ which is the number of 30minute intervals returned by the request, then all the attributes are the measurements of the intervals.

I’m rubbish at coding so the way I got a useable value is with this:

  gas_yesterday:
    friendly_name: Gas Use Yesterday
    icon_template: mdi:fire
    unit_of_measurement: 'kWh'
    value_template: "{{(state_attr('sensor.gas', 'results')[0]['consumption']+state_attr('sensor.gas', 'results')[1]['consumption']+....state_attr('sensor.gas', 'results')[47]['consumption'])|round(3)}}"

…yes, it’s a long line of code adding together all 48 values, then rounding it to 3 decimal places, but it works, and gives the total gas/electricity used in the previous day.

From what I can tell the meter sends the data for the previous day off around 0700h each day because that’s when the sensor refreshes with the next lot of data - so from 0000h to 0700h this won’t have a value because the API hasn’t caught up with the command line sensor.

I show both gas and electricity meter readings on the front end in a nice little graph showing my weekly energy usage:

05
051070×1142 39.1 KB

A much more elegant solution would be a nice little bash script that loops around and adds the dictionary array of consumption values together - if you manage this please let me have a copy of your code!

Let me know if you run into any issues.

Oh - by the way - don’t be alarmed by your gas consumption numbers, the API reports it in kWh, not metres cubed!!!

3 Likes
16 
  gas_yesterday:
    friendly_name: Gas Use Yesterday
    icon_template: mdi:fire
    unit_of_measurement: 'kWh'
    value_template: "{{(state_attr('sensor.gas', 'results')[0]['consumption']+state_attr('sensor.gas', 'results')[1]['consumption']+state_attr('sensor.gas', 'results')[2]['consumption']+state_attr('sensor.gas', 'results')[3]['consumption']+state_attr('sensor.gas', 'results')[4]['consumption']+state_attr('sensor.gas', 'results')[5]['consumption']+state_attr('sensor.gas', 'results')[6]['consumption']+state_attr('sensor.gas', 'results')[7]['consumption']+state_attr('sensor.gas', 'results')[8]['consumption']+state_attr('sensor.gas', 'results')[9]['consumption']+state_attr('sensor.gas', 'results')[10]['consumption']+state_attr('sensor.gas', 'results')[11]['consumption']+state_attr('sensor.gas', 'results')[12]['consumption']+state_attr('sensor.gas', 'results')[13]['consumption']+state_attr('sensor.gas', 'results')[14]['consumption']+state_attr('sensor.gas', 'results')[15]['consumption']+state_attr('sensor.gas', 'results')[16]['consumption']+state_attr('sensor.gas', 'results')[17]['consumption']+state_attr('sensor.gas', 'results')[18]['consumption']+state_attr('sensor.gas', 'results')[19]['consumption']+state_attr('sensor.gas', 'results')[20]['consumption']+state_attr('sensor.gas', 'results')[21]['consumption']+state_attr('sensor.gas', 'results')[22]['consumption']+state_attr('sensor.gas', 'results')[23]['consumption']+state_attr('sensor.gas', 'results')[24]['consumption']+state_attr('sensor.gas', 'results')[25]['consumption']+state_attr('sensor.gas', 'results')[26]['consumption']+state_attr('sensor.gas', 'results')[27]['consumption']+state_attr('sensor.gas', 'results')[28]['consumption']+state_attr('sensor.gas', 'results')[29]['consumption']+state_attr('sensor.gas', 'results')[30]['consumption']+state_attr('sensor.gas', 'results')[31]['consumption']+state_attr('sensor.gas', 'results')[32]['consumption']+state_attr('sensor.gas', 'results')[33]['consumption']+state_attr('sensor.gas', 'results')[34]['consumption']+state_attr('sensor.gas', 'results')[35]['consumption']+state_attr('sensor.gas', 'results')[36]['consumption']+state_attr('sensor.gas', 'results')[37]['consumption']+state_attr('sensor.gas', 'results')[38]['consumption']+state_attr('sensor.gas', 'results')[39]['consumption']+state_attr('sensor.gas', 'results')[40]['consumption']+state_attr('sensor.gas', 'results')[41]['consumption']+state_attr('sensor.gas', 'results')[42]['consumption']+state_attr('sensor.gas', 'results')[43]['consumption']+state_attr('sensor.gas', 'results')[44]['consumption']+state_attr('sensor.gas', 'results')[45]['consumption']+state_attr('sensor.gas', 'results')[46]['consumption']+state_attr('sensor.gas', 'results')[47]['consumption'])|round(3)}}"

  electricity_yesterday:
    friendly_name: Electricity Use Yesterday
    icon_template: mdi:flash
    unit_of_measurement: 'kWh'
    value_template: "{{(state_attr('sensor.electricity', 'results')[0]['consumption']+state_attr('sensor.electricity', 'results')[1]['consumption']+state_attr('sensor.electricity', 'results')[2]['consumption']+state_attr('sensor.electricity', 'results')[3]['consumption']+state_attr('sensor.electricity', 'results')[4]['consumption']+state_attr('sensor.electricity', 'results')[5]['consumption']+state_attr('sensor.electricity', 'results')[6]['consumption']+state_attr('sensor.electricity', 'results')[7]['consumption']+state_attr('sensor.electricity', 'results')[8]['consumption']+state_attr('sensor.electricity', 'results')[9]['consumption']+state_attr('sensor.electricity', 'results')[10]['consumption']+state_attr('sensor.electricity', 'results')[11]['consumption']+state_attr('sensor.electricity', 'results')[12]['consumption']+state_attr('sensor.electricity', 'results')[13]['consumption']+state_attr('sensor.electricity', 'results')[14]['consumption']+state_attr('sensor.electricity', 'results')[15]['consumption']+state_attr('sensor.electricity', 'results')[16]['consumption']+state_attr('sensor.electricity', 'results')[17]['consumption']+state_attr('sensor.electricity', 'results')[18]['consumption']+state_attr('sensor.electricity', 'results')[19]['consumption']+state_attr('sensor.electricity', 'results')[20]['consumption']+state_attr('sensor.electricity', 'results')[21]['consumption']+state_attr('sensor.electricity', 'results')[22]['consumption']+state_attr('sensor.electricity', 'results')[23]['consumption']+state_attr('sensor.electricity', 'results')[24]['consumption']+state_attr('sensor.electricity', 'results')[25]['consumption']+state_attr('sensor.electricity', 'results')[26]['consumption']+state_attr('sensor.electricity', 'results')[27]['consumption']+state_attr('sensor.electricity', 'results')[28]['consumption']+state_attr('sensor.electricity', 'results')[29]['consumption']+state_attr('sensor.electricity', 'results')[30]['consumption']+state_attr('sensor.electricity', 'results')[31]['consumption']+state_attr('sensor.electricity', 'results')[32]['consumption']+state_attr('sensor.electricity', 'results')[33]['consumption']+state_attr('sensor.electricity', 'results')[34]['consumption']+state_attr('sensor.electricity', 'results')[35]['consumption']+state_attr('sensor.electricity', 'results')[36]['consumption']+state_attr('sensor.electricity', 'results')[37]['consumption']+state_attr('sensor.electricity', 'results')[38]['consumption']+state_attr('sensor.electricity', 'results')[39]['consumption']+state_attr('sensor.electricity', 'results')[40]['consumption']+state_attr('sensor.electricity', 'results')[41]['consumption']+state_attr('sensor.electricity', 'results')[42]['consumption']+state_attr('sensor.electricity', 'results')[43]['consumption']+state_attr('sensor.electricity', 'results')[44]['consumption']+state_attr('sensor.electricity', 'results')[45]['consumption']+state_attr('sensor.electricity', 'results')[46]['consumption']+state_attr('sensor.electricity', 'results')[47]['consumption'])|round(3)}}"

Here’s the full code for the sensors that calculate the consumption total for you to copy and paste - should save you a bit of time!

1 Like
17

Thank you very much, I’ll give this a shot over the weekend! :slight_smile:

18

How did it go?

If you want to convert kWh to m3 for gas consumption you just need to divide by 11.36 (changes at very end of line 5):

  gas_yesterday:
    friendly_name: Gas Use Yesterday
    icon_template: mdi:fire
    unit_of_measurement: 'kWh'
    value_template: "{{(state_attr('sensor.gas', 'results')[0]['consumption']+state_attr('sensor.gas', 'results')[1]['consumption']+state_attr('sensor.gas', 'results')[2]['consumption']+state_attr('sensor.gas', 'results')[3]['consumption']+state_attr('sensor.gas', 'results')[4]['consumption']+state_attr('sensor.gas', 'results')[5]['consumption']+state_attr('sensor.gas', 'results')[6]['consumption']+state_attr('sensor.gas', 'results')[7]['consumption']+state_attr('sensor.gas', 'results')[8]['consumption']+state_attr('sensor.gas', 'results')[9]['consumption']+state_attr('sensor.gas', 'results')[10]['consumption']+state_attr('sensor.gas', 'results')[11]['consumption']+state_attr('sensor.gas', 'results')[12]['consumption']+state_attr('sensor.gas', 'results')[13]['consumption']+state_attr('sensor.gas', 'results')[14]['consumption']+state_attr('sensor.gas', 'results')[15]['consumption']+state_attr('sensor.gas', 'results')[16]['consumption']+state_attr('sensor.gas', 'results')[17]['consumption']+state_attr('sensor.gas', 'results')[18]['consumption']+state_attr('sensor.gas', 'results')[19]['consumption']+state_attr('sensor.gas', 'results')[20]['consumption']+state_attr('sensor.gas', 'results')[21]['consumption']+state_attr('sensor.gas', 'results')[22]['consumption']+state_attr('sensor.gas', 'results')[23]['consumption']+state_attr('sensor.gas', 'results')[24]['consumption']+state_attr('sensor.gas', 'results')[25]['consumption']+state_attr('sensor.gas', 'results')[26]['consumption']+state_attr('sensor.gas', 'results')[27]['consumption']+state_attr('sensor.gas', 'results')[28]['consumption']+state_attr('sensor.gas', 'results')[29]['consumption']+state_attr('sensor.gas', 'results')[30]['consumption']+state_attr('sensor.gas', 'results')[31]['consumption']+state_attr('sensor.gas', 'results')[32]['consumption']+state_attr('sensor.gas', 'results')[33]['consumption']+state_attr('sensor.gas', 'results')[34]['consumption']+state_attr('sensor.gas', 'results')[35]['consumption']+state_attr('sensor.gas', 'results')[36]['consumption']+state_attr('sensor.gas', 'results')[37]['consumption']+state_attr('sensor.gas', 'results')[38]['consumption']+state_attr('sensor.gas', 'results')[39]['consumption']+state_attr('sensor.gas', 'results')[40]['consumption']+state_attr('sensor.gas', 'results')[41]['consumption']+state_attr('sensor.gas', 'results')[42]['consumption']+state_attr('sensor.gas', 'results')[43]['consumption']+state_attr('sensor.gas', 'results')[44]['consumption']+state_attr('sensor.gas', 'results')[45]['consumption']+state_attr('sensor.gas', 'results')[46]['consumption']+state_attr('sensor.gas', 'results')[47]['consumption']) /11.36 | round(3)}}"
19

After seeing the value_template you created, it made me wonder if there was a way in Jinja2 to make it more compact.

At first I jumped on the idea of using a for-loop. Just iterate through all the consumption values and add them up. Unfortunately, the latest version of Jinja2 (2.10) has tightened up its scoping rules and, long story short, you can’t calculate the sum using a for-loop.

Today I saw this post and thought it was not only clever but a potential solution for making your value_template more compact.

It performs its magic using map and sum. The map creates a list containing the desired attribute values and then sum adds up all entries in the list. Very neat!

Using the Template Editor, I created some data and then tested the template.

{% set x = {"results":[{"consumption": 10}, {"consumption": 20}, {"consumption": 30}, {"consumption": 40}, {"consumption": 50} ]} %}

{{ x.results | map(attribute="consumption") | sum }}

The result is 150 as would be expected from 10+20+30+40+50.

I don’t have access to the actual data you are using so I can’t confirm that what I have to offer will instantly work for you. Here it is:

    value_template: '{{ (state_attr("sensor.gas", "results") | map(attribute="consumption") | sum / 11.36) | round(3)}}'

If it doesn’t work, please give me the data from your state_attr('sensor.gas', 'results') and I’ll make it work.

EDIT
As expected, the suggested template did contain an error and has now been corrected as per ajoyce’s suggestion.

3 Likes
20

Oh @123.

I like you a lot.

That works absolutely perfectly - being of a non-coding background I spent ages looking at python script of variables and for/sum loops and couldn’t figure out any way of making it work with a Jinja2 template.

What a beautifully simple bit of code! Many, many thanks for fixing the biggest headache in my configuration.yaml!

1 Like
21

Glad to hear it worked … and on the first try! :slight_smile:

You may have noticed I changed one other thing in the template. I added parentheses ( ) to group several terms performing the calculation. It ensures round(3) acts on the result of the entire calculation and not just on 11.36.

To see what I mean, put this into the Template Editor:

{% set x = 100.123 %}
{{ x / 3 }}
{{ x / 3 | round(1) }}
{{ (x / 3) | round(1) }}

Here are the results:

33.37433333333333
33.37433333333333
33.4

It’s all about ‘operator precedence’ and it’s spotlighted in this helpful post.

2 Likes
22

…I confess, I think there was a mismatched parenthesis from your code above as it threw an error when I first plugged it into Jinja!

    value_template: "{{ ((state_attr('sensor.gas', 'results') | map(attribute="consumption") | sum / 11.36) | round(3)}}"

throws the error, whilst

    value_template: '{{ (state_attr("sensor.gas", "results") | map(attribute="consumption") | sum / 11.36) | round(3)}}'

doesn’t (also note changes to inverted commas and quotes to make it compatible with inline templating)

But I’m being pedantic! :stuck_out_tongue:

1 Like
23

Well now that sounds more realistic! Working perfectly on the first try would be … boring! :slight_smile:

I’ve corrected my original post to reflect the change you made.

1 Like
25

Hi,

How are you doing this graph is it Garfana? I would like to do this but the code above causes an error in my config file?

Link to the topic: Graph

Thanks
Mo

26

Hi Ajoyce,

Sorry for not getting back to you with my results. I’ve just added these items into my config and whilst the config shows as valid, I’m getting results back that look odd.

From my energy tracking HS110’s I can see I used 9.29kWh of electricity yesterday:
image

The sensor you have very kindly provided shows the following instead:
image

Using the curl commands on the octopus developer website I can see the usage reflected properly there.

If you have any suggestions I’d be very grateful for your help.

Edit: my sensors to fetch the data only contain two values so I believe the fault lies with the way the data is being obtained:

  "results": [
    {
      "consumption": 0.237,
      "interval_start": "2019-04-09T00:30:00+01:00",
      "interval_end": "2019-04-09T01:00:00+01:00"
    },
    {
      "consumption": 0.244,
      "interval_start": "2019-04-09T00:00:00+01:00",
      "interval_end": "2019-04-09T00:30:00+01:00"
    }
  ],
  "friendly_name": "octopus_elec"
}

Sensor:

  - platform: command_line
    name: octopus_gas
    value_template: '{{ value_json.count }}'
    json_attributes: 
      - results
    command: >-
      curl -H '"'Authorization: Basic REDACTED'"' '"'https://api.octopus.energy/v1/gas-meter-points/REDACTED/meters/REDACTED/consumption/?period_from={{ (as_timestamp(now()) - (24*3600)) | timestamp_custom("%Y-%m-%d", True) }}T00:00:00'"'
27

Hi all,

A few updates which will hopefully answer a few of your questions.

The graph above is simply the values of the sensor plugged into the standard history-graph lovelace card. However, I’ve made some changes recently which make this whole thing work much better.

  1. The API has appeared to have only just started working properly again after being down since 06/04/2019 - even with the correct CURL requests and 200 OK responses, the latest data it was reporting was from this date. As of this afternoon service appears to be working as normal. This is almost definitely why your usage hasn’t been as expected, @anon12983141!

  2. I have found a much better way of creating a graph to chart usage. As it stands, the first sensor pulls all the consumption data from the previous 24 hours, then the second one calculates the total usage. However, the API doesn’t update with the previous day’s usage until around 6-8am (usually) which causes a 6 hour gap in the data.

The way around this that I think works the best is as follows:
First, create an input_number which will be your new consumption ‘sensor’:

input_number:
  electricity:
    name: Electricity Use Yesterday
    icon: mdi:flash
    unit_of_measurement: 'kWh'
    min: 0
    max: 100

Next, create an automation as follows:

  - alias: Electricity Update
    initial_state: True
    trigger:
      - platform: state
        entity_id: sensor.electricity
        to: '48'
    action:
      service: input_number.set_value
      data_template:
        entity_id: input_number.electricity
        value: '{{ (state_attr("sensor.electricity", "results") | map(attribute="consumption") | sum ) | round(3) }}'

Basically, this will only update the input_number with the new consumption data once it becomes available (when sensor.electricity goes from ‘0’ to ‘48’ after the API has pushed the previous day’s half-hourly consumption figures). Makes sense?

This then allows you to use @kalkih’s excellent mini-graph-card to make nice pretty energy usage charts like this:

52
521018×864 47.9 KB

  1. Some tweaks are needed to counteract daylight savings time - otherwise during the summer months the sensor will record 25 hours worth (50 records) of consumption data due to the way the API behaves. So your electricity and gas sensor commands should be updated to include both period_from AND period_to, as per the following:

curl -H '"'Authorization: Basic REDACTED'"' '"'https://api.octopus.energy/v1/gas-meter-points/REDACTED/meters/REDACTED/consumption/?period_from={{ (as_timestamp(now()) - 86400) | timestamp_custom("%Y-%m-%d", True) }}T00:00:00&period_to={{ as_timestamp(now()) | timestamp_custom("%Y-%m-%d", True) }}T00:00:00

I hope this helps with your queries! Any problems, please let me know :slight_smile:

5 Likes
28

Well that’s a nice and simple explanation! Thank you for taking the time.

29

Sorry for the lack of response i have been doing mutiple things. that is a great elpination I’m getting up to speed with HASS and loving it.

Thank you
Mo

30

Thanks for the info @ajoyce I get my smart meter installed on the 2nd of August. So look forward to getting these sensors added to my setup.