Templates in command line sensor + Octopus Energy API

[Tuxinator94 (Danny)]

Your assumption is correct. I was on a meetup in the summer where Paulus was a guest speaker and he confirmed at the moment Home Assistant only allows real time data and there is no feature to backdate readings yet.

[DaveIr (David Irwin)]

Thanks for the info. I will continue to use the great solution @oneofthemany has created and wait for the capability to get better.

[klogg]

Thanks oneofthemany this has all been really useful and I have finally found time to start setting up my config.

I have one small issue/question.

I don’t seem to be able to get 48 readings for electricity until two days later. Can you think of a reason for this? I’m suspecting it must be some quirk it the data collection for my meter because this:

https://api.octopus.energy/v1/electricity-meter-points/{mpan}/meters/{serial}/consumption/?period_from=2021-11-13T00:23:30&period_to=2021-11-13T23:59:59

returns this (the date/time now is 2021-11-14T09:00:00

{"count":0,"next":null,"previous":null,"results":[]}

But this:

https://api.octopus.energy/v1/electricity-meter-points/{mpan}/meters/{serial}/consumption/?period_from=2021-11-12T23:30:00&period_to=2021-11-12T23:59:59

returns this:

{"count":1,"next":null,"previous":null,"results":[{"consumption":0.099,"interval_start":"2021-11-12T23:30:00","interval_end":"2021-11-13T00:00:00"}]}

So apparently there is no reading for that last half an hour (unless I wait another 24 hours).

Thanks for any insight…

---

And as an aside, I am far from an expert when it comes to date time formats and as you omitted them I’m probably wrong but I just thought I’d check; shouldn’t the datetimes have a ‘Z’ at the end to cater for BST? (i.e. 2021-11-14T09:00:00Z) - it’s six months until I can check for myself!!

[oneofthemany]

yes it will take 48hrs to pull the data from the API and then you should be OK - you have to be patient

[klogg]

@oneofthemany

yes it will take 48hrs to pull the data from the API

Ah… ok. Thanks.

But doesn’t that then mean the data is for the day before yesterday?
In which case shouldn’t the rest sensors be - timedelta(days = 2)

[oneofthemany]

apologies

it has to take 48 readings which can take in excess of 24hrs

sorry for the confusion

[velkrosmaak]

How are you getting from this stage of having a well formed API call with some results, to graphs? If I curl the URL my code generates at the command line I can see results. The same URL in my restul sensor implementation I get a key definiton error because results does not exist.

[oneofthemany]

read from here in this post:

Templates in command line sensor + Octopus Energy API

Here is the code to add to the sensor.yaml file - you just need to edit the Region, MPRN,MPAN and serials: ## octopus go rest template ## - platform: rest name: Go rates today resource_template: >- {% set region = 'H' %} {% set from = as_timestamp(as_timestamp(now())|timestamp_custom('%Y-%m-%d') + 'T00:00:00') %} {% set to = from + 24 * 60 * 60 %} {% set format = '%Y-%m-%dT%H:%M:%SZ' %} https://api.octopus.energy/v1/products/GO-21-05-13/electricity-tariffs/E-1R-GO-21-05…

[velkrosmaak]

ah ok - well that is what I have been using but without getting any results returned. Having read about the potential 24+ hour delay in posts above I waited for a week but the gas or electricity sensor still weren’t returning any data.

I copied the resulting API URL from the code from this post and curl’d it (with appropriate auth) to get no results returned. If I changed the date range to the day before I get results so the command and URL in itself are correct, just the date range isn’t. Am I doing something stupid?

[DaveIr (David Irwin)]

@velkrosmaak

I copied the resulting API URL from the code from this post and curl’d it (with appropriate auth) to get no results returned. If I changed the date range to the day before I get results so the command and URL in itself are correct, just the date range isn’t. Am I doing something stupid?

Most likely you have the MPAN and Meter serial numbers wrong… That was a mistake I made for a while.
Check out the Octopus developer portal to confirm you are using the right ones:

My account | Octopus Energy

Also there are a couple of examples you can use there to test you are seeing correct data (without the date parameters used by these templates)

[oneofthemany]

Yeah this is totally possible!

@velkrosmaak : could you push some output data based on your config if the above didn’t solve your issue and we can see about getting you sorted if you’re still struggling to get results

[screzwell (Scott)]

Hi
Thanks @oneofthemany for the solution.

Is there also a 24/48hr delay in selecting Consumed Energy (kWh) on the Energy Dashboard? I’m not seeing any options.
I had to do a tweak to my yaml this morning (API to Base64) and now are seeing data for all the entities created apart from Electricity Cost, Gas Cost and Go Current Rate. I’d have thought once data is pulled through on all the consumption entities though they’d be ready to choose from?

[oneofthemany]

yeah the Base 64 seems to be missed by quite a few people

the costs only show up once you have enough data - this can take up to 2 days (highly dependent on the time you initiated your first API call

[velkrosmaak]

@DaveIr

Most likely you have the MPAN and Meter serial numbers wrong

I was hopeful this was the case, but alas I annoyingly did have them the right way around.

@oneofthemany this was just the script copied from your post unedited apart from my account details and API key. The sensors for electricity/gas readings don’t actually appear in HASS either.

I modified both scripts like this, as I was getting results when curling yesterday’s dates via the API.

- platform: rest
  name: Gas consumption
  resource_template: >-
    {% set mpan = '00000' %}
    {% set serial = 'E00000' %}
    {% set date = as_timestamp(now() - timedelta(days = 1))|timestamp_custom('%Y-%m-%d') + 'T00:00:00' %}
    {% set datefrom = as_timestamp(now() - timedelta(days = 3))|timestamp_custom('%Y-%m-%d')  + 'T00:00:00' %}
    https://api.octopus.energy/v1/gas-meter-points/{{ mpan }}/meters/{{ serial }}/consumption/?period_from={{ datefrom }}&period_to={{ date }}
  headers:
    Authorization: Basic <base 64 encoded api key>
  json_attributes:
    - "results"
  value_template: "{{ value_json.results|sum(attribute='consumption')|round(3) }}"
  unit_of_measurement: 'kWh'
  device_class: energy

If I use the curl example on the Octopus API page right now (0100 on 18/12/2012) I get results up until 2021-12-17T00:00:00Z. I don’t get that in HASS though so that I can graph yesterdays readings.

[screzwell (Scott)]

Yeah costs are still not in there yet, but strangely the off peak cost is? I’m more interested in consumption though so that is no big deal.

I’m seeing a lot of log errors relating to REST sensors in the logs, however the kWh is pulling through correctly to the entities. I just can’t select any of the consumption entities when setting up the energy dashboard so something is not quite right. I’ve created my own graphs using the data for now, just can’t understand why the Energy section can’t see them?

For now too I followed a link to convert REST to a template sensor from when REST wasn’t supported, so i have an entity called “Electricity Consumption NEW” which can be picked up by the energy dashboard. I’ve only done this for the main Elec and Gas and…i could do it for all of the different consumptions…however i’d like to clear the errors in the log but i’m not technical enough to know what they mean.

[oneofthemany]

@screzwell have you added them to the customize.yaml as per here?:

Templates in command line sensor + Octopus Energy API

well you need the following files located in your config folder: sensor.yaml customize.yaml Then simply follow the steps above and you should be good to go: I also answered most questions that would come out of the above guide to @DaveIr That’s about it

[screzwell (Scott)]

Yes I’ve followed it to the tee, apart from the secrets base64 which I edited later.
All sensors from the customise.yaml are showing in Dev Tools with a value next to them, they’re just not there in the Energy Dashboard consumption dropdown for selection. As soon as I make a template sensor from them that is selectable. I’ve made a card for all the ones created and apart from the two costs as mentioned earlier all others are populated.
I’m on 2021.12.1

[oneofthemany]

customise or customize?

[screzwell (Scott)]

customize.yaml. Was typing on mobile and auto corrected.

[oneofthemany]

without seeing your config i cant offer any further comments

please drop what you have done in the chat and perhaps we can identify the issues you seem to be having