Templates in command line sensor + Octopus Energy API

Glad to hear it worked … and on the first try! :slight_smile:

You may have noticed I changed one other thing in the template. I added parentheses ( ) to group several terms performing the calculation. It ensures round(3) acts on the result of the entire calculation and not just on 11.36.

To see what I mean, put this into the Template Editor:

{% set x = 100.123 %}
{{ x / 3 }}
{{ x / 3 | round(1) }}
{{ (x / 3) | round(1) }}

Here are the results:

33.37433333333333
33.37433333333333
33.4

It’s all about ‘operator precedence’ and it’s spotlighted in this helpful post.

2 Likes

…I confess, I think there was a mismatched parenthesis from your code above as it threw an error when I first plugged it into Jinja!

    value_template: "{{ ((state_attr('sensor.gas', 'results') | map(attribute="consumption") | sum / 11.36) | round(3)}}"

throws the error, whilst

    value_template: '{{ (state_attr("sensor.gas", "results") | map(attribute="consumption") | sum / 11.36) | round(3)}}'

doesn’t (also note changes to inverted commas and quotes to make it compatible with inline templating)

But I’m being pedantic! :stuck_out_tongue:

1 Like

Well now that sounds more realistic! Working perfectly on the first try would be … boring! :slight_smile:

I’ve corrected my original post to reflect the change you made.

1 Like

Hi,

How are you doing this graph is it Garfana? I would like to do this but the code above causes an error in my config file?

Link to the topic: Graph

Thanks
Mo

Hi Ajoyce,

Sorry for not getting back to you with my results. I’ve just added these items into my config and whilst the config shows as valid, I’m getting results back that look odd.

From my energy tracking HS110’s I can see I used 9.29kWh of electricity yesterday:
image

The sensor you have very kindly provided shows the following instead:
image

Using the curl commands on the octopus developer website I can see the usage reflected properly there.

If you have any suggestions I’d be very grateful for your help.

Edit: my sensors to fetch the data only contain two values so I believe the fault lies with the way the data is being obtained:

  "results": [
    {
      "consumption": 0.237,
      "interval_start": "2019-04-09T00:30:00+01:00",
      "interval_end": "2019-04-09T01:00:00+01:00"
    },
    {
      "consumption": 0.244,
      "interval_start": "2019-04-09T00:00:00+01:00",
      "interval_end": "2019-04-09T00:30:00+01:00"
    }
  ],
  "friendly_name": "octopus_elec"
}

Sensor:

  - platform: command_line
    name: octopus_gas
    value_template: '{{ value_json.count }}'
    json_attributes: 
      - results
    command: >-
      curl -H '"'Authorization: Basic REDACTED'"' '"'https://api.octopus.energy/v1/gas-meter-points/REDACTED/meters/REDACTED/consumption/?period_from={{ (as_timestamp(now()) - (24*3600)) | timestamp_custom("%Y-%m-%d", True) }}T00:00:00'"'

Hi all,

A few updates which will hopefully answer a few of your questions.

The graph above is simply the values of the sensor plugged into the standard history-graph lovelace card. However, I’ve made some changes recently which make this whole thing work much better.

  1. The API has appeared to have only just started working properly again after being down since 06/04/2019 - even with the correct CURL requests and 200 OK responses, the latest data it was reporting was from this date. As of this afternoon service appears to be working as normal. This is almost definitely why your usage hasn’t been as expected, @anon12983141!

  2. I have found a much better way of creating a graph to chart usage. As it stands, the first sensor pulls all the consumption data from the previous 24 hours, then the second one calculates the total usage. However, the API doesn’t update with the previous day’s usage until around 6-8am (usually) which causes a 6 hour gap in the data.

The way around this that I think works the best is as follows:
First, create an input_number which will be your new consumption ‘sensor’:

input_number:
  electricity:
    name: Electricity Use Yesterday
    icon: mdi:flash
    unit_of_measurement: 'kWh'
    min: 0
    max: 100

Next, create an automation as follows:

  - alias: Electricity Update
    initial_state: True
    trigger:
      - platform: state
        entity_id: sensor.electricity
        to: '48'
    action:
      service: input_number.set_value
      data_template:
        entity_id: input_number.electricity
        value: '{{ (state_attr("sensor.electricity", "results") | map(attribute="consumption") | sum ) | round(3) }}'

Basically, this will only update the input_number with the new consumption data once it becomes available (when sensor.electricity goes from ‘0’ to ‘48’ after the API has pushed the previous day’s half-hourly consumption figures). Makes sense?

This then allows you to use @kalkih’s excellent mini-graph-card to make nice pretty energy usage charts like this:

  1. Some tweaks are needed to counteract daylight savings time - otherwise during the summer months the sensor will record 25 hours worth (50 records) of consumption data due to the way the API behaves. So your electricity and gas sensor commands should be updated to include both period_from AND period_to, as per the following:

curl -H '"'Authorization: Basic REDACTED'"' '"'https://api.octopus.energy/v1/gas-meter-points/REDACTED/meters/REDACTED/consumption/?period_from={{ (as_timestamp(now()) - 86400) | timestamp_custom("%Y-%m-%d", True) }}T00:00:00&period_to={{ as_timestamp(now()) | timestamp_custom("%Y-%m-%d", True) }}T00:00:00

I hope this helps with your queries! Any problems, please let me know :slight_smile:

5 Likes

Well that’s a nice and simple explanation! Thank you for taking the time.

Sorry for the lack of response i have been doing mutiple things. that is a great elpination I’m getting up to speed with HASS and loving it.

Thank you
Mo

Thanks for the info @ajoyce I get my smart meter installed on the 2nd of August. So look forward to getting these sensors added to my setup.

@ajoyce Thanks for great examples / templates above.

Never having dabbled in REST before its taken me all night to get working sensors and Lovelace Cards displayed but I’ve gotten there! However I’d like to know if you think the following is possible…

Currently we have a little energy monitor on our old gas and electric meters ( Loop Device) which notifies us when we have hit 50%, 75%, 90% etc of our weekly budget. When we have our new Smart Meters installed next month I’d like to try and replicate this with HA. ( The notification side I’m happy with, its the Octopus API calls I don’t have a clue…)

I know the querying “https://api.octopus.energy/v1/products/FIX-12M-19-04-01/” lists the available tariffs; and if I can pull the applicable daily usage, multiply by the rate and add the daily charge then I should be able to work out current spend ( albeit delayed by update delay). I just don’t know how to specifically pull those bits of relevant information and store them as a useable variable in HA.

I’m certainly not asking for you to do this for me; just a few helpful points if you could? Or infact if its not possible…

Thanks in advance!

Hi there! I think in most cases you may end up complicating things by using the API to access tariffs, depending on what kind of tariff you have. If you’re on a fixed rate tariff, it’s simple; you just append the calculations for the price to the automation. Using my fixed rate tariff for electricity, which is 3.52p/kWh and 16p/day standing charge, this would be:

  - alias: Electricity Update
    initial_state: True
    trigger:
      - platform: state
        entity_id: sensor.electricity
        to: '48'
    action:
      service: input_number.set_value
      data_template:
        entity_id: input_number.electricity
        value: '{{ (((state_attr("sensor.electricity", "results") | map(attribute="consumption") | sum )*3.52)+16) | round(3) }}'

You can get your rate and standing charges from your last statement in your online account. I’ll have to give variable rate tariffs some thought, but is this the sort of thing you’re after?

Ah good point; well presented!

Thanks for that; I’ll give it a go when I get home!

Apologies for the late reply; just an update to say managed to get the cards updated and loading correctly in Lovelace!

Just waiting on the actual smart meter install to confirm working! But thank you!

So this is going to be a bit of a odd situation.

On the 28th of last month, my electricity usage stopped updating. On the 3rd of this month, my gas stopped updating. No changes have been made to my sensor.yaml entries.

The curl commands on the octopus energy developers site return valid, accurate data still (up to the dates mentioned).

Anyone else experiencing issues?

I’m having the same issue too. The curl requests work for me, but only for fetching data older than the dates you’ve specified above. Looks like the problem isn’t with this coding but instead with Octopus’s backend - they haven’t updated their database with the latest meter readings. I recall this has happened a few times in the past and normally takes a week or two for them to realise the issue and fix it.

When it is fixed, one other change you might want to make to your code is changing the end time to 23:59:59 like this:

curl -H '"'Authorization: Basic REDACTED'"' '"'https://api.octopus.energy/v1/gas-meter-points/REDACTED/meters/REDACTED/consumption/?period_from={{ (as_timestamp(now()) - 86400) | timestamp_custom("%Y-%m-%d", True) }}T00:00:00&period_to={{ (as_timestamp(now()) - 86400) | timestamp_custom("%Y-%m-%d", True) }}T23:59:59'"'

This is because at times the 0000h - 0000h time interval can return 48 OR 49 results, which breaks updates using the automation. Using the above code should guarantee 48 results every time.

Here’s hoping they fix their data upload problems soon!

1 Like

You’re good at this, you know that?

Thank you for your speedy and thorough response!

1 Like

Always a pleasure! Have also tweaked this integration so it’s in a bash script to keep the main config.yaml tidy - can share this if needed.

Anyway - I see yesterday my gas and electricity sensors kicked back into gear again, looks like they’ve fixed things on the backend - are you seeing success with yours?

Started getting new data but there’s still a large gap unfortunately. Hopefully they sort themselves out.

I’d be more than interested in seeing the bash script if you feel like sharing.

Sure. Thanks to @123 for doing the bulk of the conversion work!

config.yaml:

  - platform: command_line
    name: electricity
    value_template: '{{ value_json.count }}'
    scan_interval: 600
    json_attributes: 
      - results
    command: 'sh /config/bash_scripts/electricity.sh'
  - platform: command_line
    name: gas
    value_template: '{{ value_json.count }}'
    scan_interval: 600
    json_attributes: 
      - results
    command: 'sh /config/bash_scripts/gas.sh'

electricity.sh:

STARTDATE=`date -d @$(( $(date +"%s") - 86400)) +"%Y-%m-%dT00:00:00"`
ENDDATE=`date -d @$(( $(date +"%s") - 86400)) +"%Y-%m-%dT23:59:59"`
curl -H "Authorization: Basic xxxxxxxxxxxxxxxx" "https://api.octopus.energy/v1/electricity-meter-points/yyyyyyyyyyyyyyyy/meters/zzzzzzzzzzzzzzzz/consumption/?period_from=$STARTDATE&period_to=$ENDDATE"

gas.sh:

STARTDATE=`date -d @$(( $(date +"%s") - 86400)) +"%Y-%m-%dT00:00:00"`
ENDDATE=`date -d @$(( $(date +"%s") - 86400)) +"%Y-%m-%dT23:59:59"`
curl -H "Authorization: Basic xxxxxxxxxxxxxxxx" "https://api.octopus.energy/v1/gas-meter-points/yyyyyyyyyyyyyyyy/meters/zzzzzzzzzzzzzzzz/consumption/?period_from=$STARTDATE&period_to=$ENDDATE"

Replace x’s, y’s and z’s as required and link to wherever you’ve put your scripts - I’ve put mine in a folder called bash_scripts in my config folder (correct file path to config folder can be found in “Developer Tools -> Info -> Path to configuration.yaml”

1 Like

Hi @ajoyce I have tried to follow along with your Octopus API example, but I am not sure if I am missing parts of your config?

My setup is in a package as follows:

# Octopus API
sensor:
- platform: command_line
  name: electricity
  value_template: '{{ value_json.count }}'
  scan_interval: 600
  json_attributes: 
    - results
  command: >-
    curl -u "APIKey:" "https://api.octopus.energy/v1/electricity-meter-points/MPN/meters/SERIAL/consumption/?period_from={{ (as_timestamp(now()) - 86400) | timestamp_custom("%Y-%m-%d", True) }}T00:00:00&period_to={{ (as_timestamp(now()) - 86400) | timestamp_custom("%Y-%m-%d", True) }}T23:59:59"
    
input_number:
  electricity:
    name: Electricity Use Yesterday
    icon: mdi:flash
    unit_of_measurement: 'kWh'
    min: 0
    max: 100

automation:
- alias: Electricity Update
  initial_state: True
  trigger:
    - platform: state
      entity_id: sensor.electricity
      to: '48'
  action:
    service: input_number.set_value
    data_template:
      entity_id: input_number.electricity
      value: '{{ (state_attr("sensor.electricity", "results") | map(attribute="consumption") | sum ) | round(3) }}'

Yesterday I just added the sensor without the automation and got the Blue bar with 50 (as per image below. When I added the automation this morning and restarted I got the Orange bar with 2.

I manually triggered the automation and got a consumption of 0.287kWh which I guess is the sum of the two entries from yesterday?

electricity%20sensor

Am I missing parts of the configuration out?

If i try curl -H "Authorization: Basic xxxxxxxxxxxxxxxx"

The sensor returns no values.

Are you bale to share your complete configuration?