Hi Dave,
I’m just to use your Curl command in terminal but getting a syntax error.
I’m trying to adapt this for Webcore so I can pull the current result and (for the sake of SmartThings and Homekit) make it apply to a virtual dimmer switch.
Could anyone help?
I’m about to change supplier, considering this. Aside from the ‘leading edge’ automation interest how happy are you with this tariff and approach , and I guess supplier but Octopus seem ‘OK’
Also, and you may not have this information, if you hadn’t, or couldn’t change your usage pattern how do you feel the tariff would have fared ?
I have about 1.5KWHr constant usage 24hrs and then the usual household additions during an offset daytime of noon to midnight. £2500 annually - too much tech ! I’m thinking if going without management that might benefit me as a disproportionate offpeak usage. But I don’t have Economy 7 (gas here)
@xAPPO I’ve been impressed with them so far, and the Agile Octopus tariff itself. There is a tool near the bottom of this page - https://octopus.energy/agile/ - which shows general usage patterns and shows that you should still save.
I would say we’ve managed to move when we use the dishwasher and washing machine, but haven’t made many other changes and it is still working out a good amount cheaper than what our previous supplier would have charged. I’ve definitely enjoyed writing the octoblock and octocost apps to work in Home Assistant and then use them to trigger certain things to turn on, like charging the cordless hoover, plus having information on running costs. We are also on gas heating.
Also their referral is pretty good, if you use my link - https://share.octopus.energy/dense-ape-125 - we’d each get £50 credit.
If I do of course I will as I appreciate people who contribute such code.
Looking at the home based battery storage solutions , not specifically Tesla PowerWall , do you think they have any commercial merit here ? Even on near break even as it satisfies a tech rush for me.
Covering that 24hr 1.5KWHr at lowest possible rate I guess is the best approach. I know … turn things off but I like flashing LED’s.
Being a listed building solar has not been an option, nor double glazing… but rates come in abundance.
I definitely think a battery storage solution would be interesting. It is something I’m likely to look at in the future, I just haven’t had the time (or finance) to do it at the moment. Either to charge up when the price is low, and then use for the house when the price is high, or possibly sell back to the grid when the price is high, as there is also an Outgoing Agile tariff to make the most of selling back to the grid when the price is high.
I had a very quick look at https://www.powervault.co.uk/ but I need to do more research.
Finally got a solution working for a local dashboard with today’s Octopus Agile pricing.
The graph shows a rolling 24Hr so far left bar is the current price. When it gets beyond half 10 at night you start to see a blank area to the right as the new data does not come out until about 16:30 but that is fine for my purposes. It is easy for the rest of the household to understand.
This is a Node-Red dashboard as I could not get HADashboard to authenticate and let me show the Grafana Graph.
Data is pulled into Infludb from the API by NR, the graph is produced by Grafana. Couldn’t have happened without @frenck and all of his add-ons.
[edit]
I realised I had missed off explaining a number key things I discovered to make this work.
In the add-on options
plugins: []
env_vars:
- name: GF_AUTH_ANONYMOUS_ENABLED
value: 'true'
- name: GF_SECURITY_ALLOW_EMBEDDING
value: 'true'
- name: GF_AUTH_ANONYMOUS_ORG_NAME
value: XXX.XXX.net
ssl: true
certfile: fullchain.pem
keyfile: privkey.pem
In Grafana add in an organisation to match the one above.
Truth be told, I fiddled and added all sorts of things to try and make it work. It now does so I am not fiddling again!
To display the graph without controls change the /d/ in the URL to /d-solo/.
In the Webpage Card Configuration add an &refresh=5m to get it to refresh and also a &theme=light to match standard Lovelace UI.
https://xxx.xxxx.net:3000/api/hassio_ingress/XXXX/d-solo/7p1LkuRgk/agileprice?orgId=2&panelId=2 width="200" height="100" frameborder="0"&refresh=5m&theme=light
&refresh=5m to end of URL as per Lovelace UI[{"id":"7dd32ba9.631134","type":"ui_template","z":"7c1c1dc2.f60624","group":"e453ab5b.693c08","name":"Agile Pricing","order":1,"width":0,"height":0,"format":"<div style=\"border:2px; overflow:hidden;\">\n\n <iframe \n id=\"form-iframe\" \n src=\"https://xxx.xxx.net:3000/api/hassio_ingress/PLuAAK0XIwRorNFRT_oFqbWiUB746HpLKLvw0qM4ocE/d-solo/7p1LkuRgk/agileprice?orgId=2&panelId=2&refresh=5m\"\n style=\"margin:0; width:100%; height:250px; border:none; overflow:hidden;\" \n scrolling=\"no\" \n onload=\"AdjustIframeHeightOnLoad()\">\n </iframe>\n \n <script type=\"text/javascript\">\n function AdjustIframeHeightOnLoad() \n { document.getElementById(\"form-iframe\").style.height = document.getElementById(\"form-iframe\").contentWindow.document.body.scrollHeight + \"px\"; }\n function AdjustIframeHeight(i) \n { document.getElementById(\"form-iframe\").style.height = parseInt(i) + \"px\"; }\n </script>\n</div>\n\n\n","storeOutMessages":true,"fwdInMessages":true,"templateScope":"local","x":801.25,"y":45,"wires":[[]]},{"id":"e453ab5b.693c08","type":"ui_group","z":"","name":"Group 1","tab":"229ad979.0c7e56","order":1,"disp":false,"width":"8","collapse":false},{"id":"229ad979.0c7e56","type":"ui_tab","z":"","name":"Octopus","icon":"dashboard","order":1,"disabled":false,"hidden":false}]
If your base load is 1.5kWh do you have electric heating or something?
I would seriously look at reducing that base load first!
Simple things like leaving an electric oven switched on (I don’t mean the fan running, purely just the clock) uses about 30w an hour. That’s about £36 a year!
If you can’t get your base load down perhaps the OctopusGO tariff might be better?
You can charge your house battery from 00:30 - 04:30 at 5p per kWh and then use that to supplement the rest of the day.
The problem you will face with such a high base load is the fact you will need 18kWh of storage just to cover your base load.
I have solar and a single 4.5kWh battery but the following pic gives an idea how it helps out with energy usage from the grid. With a second battery I could go 24/7 with virtually no grid pull when it’s sunnier weather.
I am using SolaX Inverter and Battery Storage.
Hey Baz, your 24 hour graph looks exactly like what i’m missing - I have historic data in Influx thanks to a really handy Python tool called Octograph and I’ve used a REST sensor you described to have a now and next price in the HA frontend but I’d really like to graph future prices so we can glance at a graph and know when is best to plan heavy energy use. I can’t find your node-red contribution, guessing it hasn’t been released to NPM?
No it is just in my own repo. You can add it directly using the options in the add-on. I posted a link earlier in the thread. Octopus Energy Agile Tariff - #32 by baz123. Bit rough and ready right now. I’m going to include the data formatting for influx as a third output at some point. But for now you can use the flow above.
I gave up on that in preference to the output from my node.
Hi Brian,
Do you mind sharing how do you process the raw json output to that graph? I’m using your node but not sure how do I query the data from influx to grafana to get that same graph.
Firstly I process the data thus (from the first output from my node and pass it to a influxdb out node.
[{"id":"b6058e53.fd887","type":"function","z":"8dea1bec.971898","name":"Format for InfluxDB","func":"var newmsg = {};\nnewmsg.payload = [];\n\nmsg.payload.results.forEach(myFunc);\n\nfunction myFunc(item, index) {\n newmsg.payload.push([{ value_inc_vat : item.value_inc_vat, \n \"time\": new Date(item.valid_from).getTime() *1000 *1000}, {\"source\" : \"Agile\"}]);\n}\n\nnewmsg.measurement = \"OctopusPrice\"\n\nreturn newmsg;","outputs":1,"noerr":0,"x":470,"y":300,"wires":[["f61144e.fc690b8","79898d8d.cddcc4"]]}]
You don’t then need a query in InfluxDB just Grafana (linked to the db of course).
First time try to export JSON from Grafana - how does this work?
{
"datasource": null,
"aliasColors": {},
"bars": true,
"dashLength": 10,
"dashes": false,
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 10,
"w": 7,
"x": 0,
"y": 0
},
"hiddenSeries": false,
"id": 2,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": false,
"total": false,
"values": false
},
"lines": false,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"dataLinks": []
},
"percentage": false,
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"groupBy": [],
"measurement": "OctopusPrice",
"orderByTime": "ASC",
"policy": "default",
"query": "SELECT \"value_inc_vat\" FROM \"OctopusPrice\" WHERE $timeFilter",
"rawQuery": true,
"refId": "A",
"resultFormat": "time_series",
"select": [
[
{
"params": [
"value_inc_vat"
],
"type": "field"
}
]
],
"tags": []
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Agile Pricing",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
}
Time range
When you try and share the panel deselect the current time range option so it updates.
That json worked like a charm, thanks!
A big thanks to contributors to this thread. I now have my Agile integration working nicely: see below.
The “Cheapest times” entity card is generated from a Python script that queries the Octopus API and builds a matrix of costs averaged over each period, finds the cheapest and dumps the result as a JSON string.
HA calls this script as a command-line sensor on the half-hour (triggered with an automation as described in a prior post), and has template sensors to pull the individual values out.
Next step is to automate my hot water system to choose whether to use gas or the immersion depending on cost, and perhaps carbon intensity (gas is allegedly about 215g/kWh from a decent domestic boiler, whereas the grid can hit over 350g on a bad day).
Looks good. How are you pulling the grid intensity and from where? I’ve got a REST sensor setup but I’m not overly happy with it.
I really wish there was a better gauge in HA or Node-RED.
Here’s my sensor. Documentation, including region IDs (pasted below), is here.
- platform: rest
resource: https://api.carbonintensity.org.uk/regional/regionid/8
name: "West Midlands carbon intensity"
unit_of_measurement: 'g/kWh'
value_template: '{{ value_json.data.0.data.0.intensity.forecast }}'
scan_interval: 600
1 North Scotland
2 South Scotland
3 North West England
4 North East England
5 South Yorkshire
6 North Wales, Merseyside and Cheshire
7 South Wales
8 West Midlands
9 East Midlands
10 East England
11 South West England
12 South England
13 London
14 South East England
15 England
16 Scotland
17 Wales
Thanks - almost exactly the same.
Have you found anywhere that tells you the boundaries for their very low / low / medium etc values?
I pull off the intensity as the state and the forecast as an attribute.
- platform: rest
name: Carbon Intensity
resource: https://api.carbonintensity.org.uk/regional/regionid/2
value_template: '{{ value_json.data[0].data[0].intensity.index }}'
json_attributes_path: "$.data[0].data[0].intensity"
json_attributes:
- "forecast"
I haven’t, no. I’d imagine this is deliberate on their part to allow them to pull the category thresholds about as more renewables come online and relative performance changes.
For example, the average for 2012 was over 500g/kWh, and this was down to under 200g/kWh in 2019. Progress on the UK grid has been phenomenal and generally under-reported.
I’ve added this in now. Setting hours to 0 with start_period set to now in apps.yaml will return the current import or export price. And set sensors sensor.octopus_current_price or sensor.octopus_export_current_price as appropriate.
Here’s my script for working out the cheapest n-block cost, which I call octoplan.py. Seems to do broadly the same as @badguy’s octoblock but without needing appdaemon. It’s a bit less “integrated” as a result, of course: it’s read with a command_line sensor to give the cheapest times display as per my screenshot above; and also queried with a bash script (run via a shell_command automation) that writes the immersion start time to a text file that HA reads with another command_line sensor. Clunky, but I understand what it’s doing.
The from and to arguments need to be in recognizable time formats, and it’ll probably fail in all sorts of horrible ways if you feed it invalid arguments.
My most recent addition is the latest flag, which I use for the immersion schedule. If there are two or more periods with exactly the same cost in the time span being evaluated, the default is to return the earliest. If you specify --latest, it’ll instead return the latest block which makes more sense for an overnight immersion run.
The first line refers to octopus_lib. This is Octopus’s David Winterbottom’s client.py from here which I have downloaded and renamed octopus_lib.py to give me half a chance of remembering what it is in three years’ time. You will need to ensure you have installed the Python requests library as well as the imports below (dateutil is installed as py-dateutil).
The final section of the script generates an output list of time/cost dictionaries of cheapest times averaged over 30m in output[0] to 4h in output[7]. The final line prints out the JSON for 30m, 1h, 2h and 4h only, but you can select others if needed.
If you use it and it breaks, you get to keep all the pieces. Please be considerate in querying Octopus’s API: I run this half-hourly via an automation trigger (see earlier posts).
from octopus_lib import APIClient
import argparse
import datetime
import json
from dateutil import parser
from dateutil.tz import tzlocal
APIKEY = 'YOUR_API_KEY'
REGIONCODE = 'E' # replace with your region code
argparser = argparse.ArgumentParser(description='Finds cheapest Agile chunks')
argparser.add_argument('-f',
'--from',
help='Start time for search',
required=False)
argparser.add_argument('-t',
'--to',
help='End time for search',
required=False)
argparser.add_argument('-l',
'--latest',
help='Find latest rather than earliest occurrence',
action='store_true',
required=False)
args = vars(argparser.parse_args())
from_ts = None
to_ts = None
if args['from'] is not None:
from_ts = parser.parse(args['from']).astimezone(tzlocal())
if args['to'] is not None:
to_ts = parser.parse(args['to']).astimezone(tzlocal())
latest = args['latest']
# pull in the rates
octo = APIClient(APIKEY)
now = datetime.datetime.now()
rates = octo.agile_tariff_unit_rates(REGIONCODE, period_from=now)
# API returns rates as latest-first, fix that...
rates['results'].reverse()
# Extract costs and times into lists
costs = []
times = []
for result in rates['results']:
period_str = result['valid_from']
time_ts = parser.parse(timestr=period_str).astimezone(tzlocal())
if from_ts is not None and time_ts < from_ts:
# start specified and not yet reached: continue searching
continue
if to_ts is not None and time_ts >= to_ts:
# end specified and reached: break out
break
times.append(time_ts)
costs.append(result['value_inc_vat'])
# Build a matrix of half-hour periods (vertical) and 1-8 averaging periods
# (horizontal) (so second column is hour average costs)
cost_matrix = []
for x in range(len(costs)):
cost_row = [costs[x]]
for y in range(1, 8):
if len(costs) - x < y + 1:
# table ends before averaging period
cost_row.append(None)
else:
cost_total = 0
for z in range(y + 1):
cost_total = cost_total + costs[x + z]
cost_row.append(round(cost_total / (z + 1), 2))
cost_matrix.append(cost_row)
output = []
for x in range(8):
# Build column, find minimum
cost_col = []
for y in range(len(costs)):
if cost_matrix[y][x] is not None:
cost_col.append(cost_matrix[y][x])
if latest:
mindex = len(cost_col) - 1 - cost_col[::-1].index(min(cost_col))
else:
mindex = cost_col.index(min(cost_col))
output.append({"time": "%s" % times[mindex].isoformat(),
"cost": "%.2f" % cost_col[mindex]})
# output a list for cheapest 30m, 1h, 2h and 4h
print(json.dumps([output[0], output[1], output[3], output[7]]))
Just added this in but get the following error:
2020-04-13 10:04:44.829543 WARNING octo_block_now: Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/appdaemon/threading.py", line 766, in worker
funcref(self.AD.sched.sanitize_timer_kwargs(app, args["kwargs"]))
File "/config/appdaemon/apps/octoblock/octoblock.py", line 42, in period_and_cost_callback
elif start_period == 'now':
File "/config/appdaemon/apps/octoblock/octoblock.py", line 94, in get_period_and_cost
ZeroDivisionError: division by zero