It is not unknown for people to post ‘I can’t see anything in my entities’ and then ‘oh yes they are here’. I do think that sometimes it takes a while, and then there is ‘caching’ in web browsers, and all that stuff.
This is, really, rather complex and a lot is going on, so frankly it is amazing that a) it works and b) someone else can download my Node-RED and flow and make it work for themselves.
Yes, the _2 sensors are a pain. This happens because all entities are registered with HA (created) and HA keeps the registrations for a while, and if you remove the Node-RED entity then it vanishes from HA, but when you add it back, HA already remembers it, so adds a second. (Then you patch everything in HA to _2, and then after the next reboot HA has forgotten your first attempt, then re-creates without the _2 and you have to update everything again. Deep Joy)
I sincerely hope it adds value for you.
There are other integrations and bits and pieces using Octopus Agile, but the advantage here is that you can do anything with the Node-RED flow (code).
I have moved on to a new update but it is too big to post here, so I have at some point to get around to posting it to the Node-RED forum. There have been a few issues, but mostly this all works, day after day, and pulls down the Agile Prices and does a bit of simple processing.
Good luck with your project work - I started with Solar PV two years ago and this has turned into quite a time consuming project, but I have learnt so much!
I have been playing with Home Assistant on and off for a year or so, and experimented with Node Red too.
I am on Octopus Agile. I don’t have a battery, but am taking delivery of an EV in the next couple of weeks. The use case I am looking at is setting a target state of charge and ready-to-go time for the EV, then based on the amount of energy required, picking the cheapest ‘n’ 30-minute timeslots starting now to charge the car, reaching the desired state of charge at or before the ready-to-go time.
My NodeRed experience is pretty much limited to turning stuff on when Saving Sessions occur, activating appliances when prices go negative, switching christmas tree lights on and off according to a schedule and so on…
Looks like you’ve done most of the work for me with your code, need to see whether I can get the rest of what I need over the line!
I’m up and running, with two-and-a-half automated use cases
immersion on/off in the low price periods; working nicely!
heat pump auto off at the start of the high export price periods. Still trying to decide whether to do the inverse and turn heat pump on again at the end of the high export period
car chargepoint automatically on at the start of the low price period. Weirdly, it seems my Zappi unit isn’t easy to turn off! So I either need to work out how to programmatically set end time for the (smart) boost, or rely on the car’s charge limit, or change the mode of the chargepoint to “stopped” and then back to what it was. Not ideal.
One little question - I haven’t done any definitive testing, but on the Apex chart, the previous day’s optimal periods disappear after the new rates are imported. It is likely that a high export period starts before 4pm and ends after the rates have been refreshed - in the example below (imagining I were using export array rather than import array which is the one likely to cross over the refresh time) do you expect the “off” template will still be triggered?
I think the automation code provided by @Shaggy91 sets offtime at the point in time of the template running and therefore only a failure of Home Assistant, not a refresh of the sequence table, would prevent off trigger running?
Also, is there any way to keep the previous day’s periods displayed on the Apex Chart? I’m guessing the array gets refreshed, so likely no.
alias: Immersion Agile
description: Immersion Agile price integration
trigger:
- platform: template
id: "on"
value_template: >
{% set ns=namespace() %}
{% set ns.flag=false %}
{% for event in state_attr('sensor.octopus_agile_sequence_table_2',
'import_array')%}
{% set ontime=(as_timestamp(event.from)|int-as_timestamp(now())|int) %}
{% set ns.flag = ns.flag or ( 0 < ontime < 120) %}
{% endfor %}
{{ns.flag}}
- platform: template
id: "off"
value_template: >
{% set ns=namespace(flag = false) %}
{% for event in state_attr('sensor.octopus_agile_sequence_table_2',
'import_array')%}
{% set offtime=(as_timestamp(event.upto)|int-as_timestamp(now())|int) %}
{% set ns.flag = ns.flag or (0 < offtime < 120) %}
{% endfor %}
{{ns.flag}}
condition: []
action:
- service: switch.turn_{{ trigger.id }}
target:
entity_id: switch.fstwifitu_immersion_ext2_switch_1
mode: single
I confess that I don’t actually use Agile. I have it all running and do check the display from time to time, but I don’t use it ‘in anger’ so I have never really check the end use-cases like this.
The tariff array holds 48 hours worth so it spans both up to 23:00 today, and after 16:00 also tomorrow. Yes you are correct - once the update runs the best groups and the best consecutive periods are re-calculated on the new (tomorrow) tariff part, and these then overwrite the stored context variables.
Yes, if you pre-load the on / off times as required before the 16:00 update then switching should continue over the transition period. However, I am sure this is something that could be improved.
In the short term, you can change the update trigger time to later. I originally set this to just after 16:00, which is the earliest point Octopus update the tariffs, and mostly reliable, but Octopus had a period back in the year when it was happening nearer 19:00. I have changed the code so that update runs when there are only 10 off 30 minute periods left, so as to run every half hour after say 18:00 and keep on checking until a successful update happens.
I will have a look at the code and see if I can merge the new calculation with the existing values, then trim to just today-tomorrow, and save that. This should be relatively easy, and will fix both display and availability for control - I probably need to add an extra field for “yesterday” and “today” so it is possible to tell the two combined sets of data apart!
After your post I have been mindful that I have not yet published my new version which has been tested for DST and year end (just fixed a bug in that). This update is overdue, but I have to push the code out via Node-RED forum as it is too big for this site. Let me see what I can do…
I’m not on Agile -yet- but I’m planning my automations as if I were. In the meantime, or if I stay on Intelligent Go, I figure that these automations will still reduce my household carbon profile because Agile prices reflect CO2e very closely.
I’m using the Octopus Compare app which looks at usage data for my account via the Octopus API. In the month since I had the heat pump installed, Agile would have been nearly 40% cheaper (the app doesn’t know about the intelligent cheap rate periods, only the 6 hour cheap rate periods, so it overestimates the Agile saving slightly). Incidentally Agile would have been 70% cheaper than Flexible for me this month. Flexible is not meant for high off peak users!
It’s been a windy month so this may be unusual but if I get the automations right I will be able to ditch Octopus’ version they call intelligent for the EV, and use your version for everything, thus getting a true indication of my Agile savings. Yours does seem to match up very closely with the schedules they create for the EV (unsurprisingly but reassuringly.)
FYI, I’ve changed the trigger time to 20:05 as the last “high export” period commonly finishes around 19:30 and I’d like to be able to see that it’s still active for debugging (which is commonly an evening activity for me.) Looking forward to seeing your up to date flow when you find a way to publish it. In the mean time this is great and I didn’t perceive any end of year bug!
Which controls multiple switches with caveats to charge batteries and protect elements. The offtime on the table works well. The only failure is if the switch drops off the network (on wifi) or if home assistant has a bad day. To try and put these niggles to rest i have 3 asus routers around the house in AIMesh mode and they reboot every 2 or 3 days at 0310 in the morning. This helps keep the network fresh and reduces hiccups with devices dropping off.
I’m finding that the octopus intelligent integration is going offline at random times. I have set up an automation to reload the integration when it goes “offline” or in to an “unknown” state which can work done off the times, but the other times the error log says:
Logger: homeassistant.components.automation.intelligent_reload
Source: components/automation/__init__.py:690
Integration: Automation (documentation, issues)
First occurred: 09:16:14 (3 occurrences)
Last logged: 11:13:32
Error while executing automation automation.intelligent_reload: The config entry (octopus_intelligent) with entry_id 7ecb0d4804d0d6fa151bb98d91e79aca cannot be unloaded because it is not in a recoverable state (ConfigEntryState.FAILED_UNLOAD)
Has anyone else experienced this and has a better work Around? I also checked another friend HA instalation when my intelligent integration was offline in this state, his was still working OK…
I’m trying a different way (I’m not on Agile yet so I can’t use the Octopus sensors that you have). However I’ve run out of coding skill. The data I want does exist and below is what ChatGPT suggests to only run if the average is above 15p/kWh (the minimum average price I want to export) but I’m not convinced the arrays will match in the enumeration - any thoughts please?
Yeah the arrays don’t match so that didn’t work. Two ways I was overcomplicating
Every day I’ve observed, the higher price peak is in the afternoon, so I can just add a condition to ignore the morning. Only thing is, I’m not sure if this will always be true on sunny days in the summer. Time will tell I guess.
condition:
- condition: time
after: "15:00:00"
before: "19:30:00"
weekday:
- mon
- tue
- wed
- thu
- fri
- sat
- sun
We’re already in the export array, so it’s simpler to use the export prices as a proxy for the import prices, and just adjust the trigger event value accordingly, then find the average of the next few events. I think I have an appropriate template with conditions and I’m testing it.
The first code below works, using export array as a proxy for high import price/high carbon to turn off power-intensive device(s), in my case all AC zones.
What mistake have I made with the template below it, which attempts to use the import prices sensor directly? I realise that would trigger for any half-hour period, not for a contiguous block but I would like to experiment with both (rather than deviate from Geoff’s node-RED flow and add a high-import prices block to the array, which would be yet another alternative I may yet pursue. I read that there are limits on array size though…)
The message from Developer Tools: Template for the second set of code is
TypeError: ‘NoneType’ object is not iterable
alias: Agile Export peak above slider value
description: >-
Agile high price/carbon turn AC off and create notification (test for slider,
works with value)
trigger:
- platform: template
id: "off"
value_template: >
{% set ns=namespace() %}
{% set ns.flag=false %}
{% for event in state_attr('sensor.octopus_agile_sequence_table_2',
'export_array') %}
{% set ontime=(as_timestamp(event.from)|int-as_timestamp(now())|int) %}
{% if 0 < ontime < 120 and event.average | float >= states('input_number.agile_export_target') | float %}
{% set ns.flag = true %}
{% endif %}
{% endfor %}
{{ ns.flag }}
condition: []
action:
- service: climate.set_hvac_mode
data:
hvac_mode: "off"
target:
entity_id:
- climate.bedroom_2_c
- climate.bedroom_3_d
- climate.downstairs_open_plan
- climate.master_bedroom_a
- climate.study_b
- service: persistent_notification.create
metadata: {}
data:
message: >-
AC turned off because average for upcoming period exceeded export target
slider value. Adjust the export target slider if action desired.
title: High carbon:AC off!
- service: notify.whatsapp
metadata: {}
data:
message: >-
Price/carbon optimisation active: AC off! Average for upcoming period
exceeded export target slider value.
mode: single
Note: number helper slider defines the export target in the code above; for testing simplicity it’s hard-coded in the one below.
{% set ns=namespace() %}
{% set ns.flag=false %}
{# We use the calculated high export period to turn *off* some high-current devices, saving money on Agile and/or reducing carbon load by proxy #}
{% for event in state_attr('sensor.octopus_agile_prices_2', 'array') %}
{% set ontime=(as_timestamp(event.from)|int-as_timestamp(now())|int) %}
{# Templates run each minute, so allow 60 secs each way, then check if the average is above the target set by a number helper (inconvenient if low impact) #}
{% if 0 < ontime < 120 and event.import >= 18 %}
{% set ns.flag = true %}
{% endif %}
{% endfor %}
{{ ns.flag }}
Edit: corrected the capitalisation of array as per Geoff’s advice
When i check the state in the developer tools, it says unavailable, however if i put a debug node on the end of the pricing flow i do see an entity with 96 attributes.
I’m quite new to this - this is my first delve into Node-Red so its quite possible im doing/have done something wrong! So any advice is much appreciated
After a quite a lot of googling, tinkering and trying stuff out, i think i have found the solution. It came down to the sensor configuration. The state class of measurement was the issue, if you clear that down and restart it the problem goes away.
Alas, my original over-exuberant use of State class ‘measurement’ hit problems last year when the rules were tightened.
Although this was picked up at the time, it was remiss of me not to try to edit my original post (I don’t think I was aware that this was actually possible at the time).
I have now re-loaded the original Node-RED flow (in first post) with slight modifications to clear the offending State Class entries.
Hard to believe that it is one year since I first posted this project…
Although I still don’t use Octopus Agile tariff myself, I continue to look at the graphical display of agile prices, and have been tinkering with the code ever since. Last summer I attempted to add DST and local time. I waited until the time-change in October to test it, and I did for the first time in my life get up in the night just to watch the clocks change. The code did, for the most part, actually work.
From discussions on this thread, I have added a binary sensor to act as a switch. Updates to the display (graph, tables) as well as general improvements to the JSONata code itself.
The basic JSON file for the Node-RED flow is now too large to post here. I have finally got round to learning Github and all that entails, so I am now officially an old git. Yet more learning achieved.
So, the latest update is now on Github. I have collated the documentation into the read.me file, and tested the release, so time to commit.
Hi Geoff - I don’t know if you’d prefer this feedback/ask on github or here… but as I’ve already posted about a different issue there, I’m starting here with this one until you say different
First of all thank you for such an amazing job in terms of intellectual effort AND such helpful writeups and support to people who’ve come unstuck, and all whilst turning down ‘buy me a coffee’ - what an amazing bloke.I am just struggling to get the entity sensor.octopus_agile_sequence_table to display anywhere* and have just discovered the following warnings repeating in nodered:
“Deprecated API warning: Calls to RED.util.evaluateJSONataExpression must include a callback. This will not be optional in Node-RED 4.0. Please identify the node from the following stack and check for an update on npm. If none is available, please notify the node author.”
18/02/2024, 18:08:41msg : string[1378]
</>
“Error: ↵ at Object.evaluateJSONataExpression (/opt/node_modules/@node-red/util/lib/util.js:775:18)↵ at JSONataService.evaluate (/config/node_modules/node-red-contrib-home-assistant-websocket/dist/common/services/JSONataService.js:58:39)↵ at TypedInputService.getValue (/config/node_modules/node-red-contrib-home-assistant-websocket/dist/common/services/TypedInputService.js:58:85)↵ at SensorController.onInput (/config/node_modules/node-red-contrib-home-assistant-websocket/dist/common/controllers/SensorBaseController.js:76:40)↵ at SensorController._InputOutputController_preOnInput (/config/node_modules/node-red-contrib-home-assistant-websocket/dist/common/controllers/InputOutputController.js:62:76)↵ at /opt/node_modules/@node-red/runtime/lib/nodes/Node.js:214:26↵ at Object.trigger (/opt/node_modules/@node-red/util/lib/hooks.js:166:13)↵ at Sensor.Node._emitInput (/opt/node_modules/@node-red/runtime/lib/nodes/Node.js:206:11)↵ at Sensor.Node.emit (/opt/node_modules/…”
</>
18/02/2024, 18:08:42msg : string[270]
Have I broken something or is there a direction of investigation you could recommend please?
*example symptom:
apexcharts-card custom card attempts produce:
Entity not available: sensor.octopus_agile_sequence_table
Entity not available: sensor.octopus_agile_sequence_table
I have seen your messages on Github, I’m just not able to respond easily at the moment.
The issue you quote above is most likely due to out of date Websocket nodes. With Node-RED v3+ all JSONata calls now expect a callback function, which was indeed added in the HA Websocket nodes in a recent update. If you are on the latest HA core, the latest NR addon, the latest NR companion integration and the latest Websocket nodes, you should not see this error and it should all work.
I suggest checking your versions. BUT DO BE CAREFUL. There have been several breaking changes across these components, and incompatible updates may break your system.
As a start, I suggest:
Take an inventory of your versions, and ideally save a backup copy and remove your old (this) Node-RED flow completely (delete, restart HA and NR), before installing again afresh. So much has changed since my first flow that it is probably better to start again rather than try and copy over the old flow - but you will need the latest versions.
Thank you Geoff. I backed up this morning, and have inventoried as you kindly suggested:
HA Core 2024.1.6 vs 2024.2.2
NR Addon 17.0.7 - current
NR companion 3.1.2 vs 3.1.3
Websocket nodes 0.55.1 vs 0.63
so you were right on the money with your diagnosis. I disabled config nodes, deleted the whole flow, restarted, re installed, re configured. Whenever I try this I do run into nodered alerting me to pre-existing nodes, and I’ve not yet taught myself how to delete these from homeassistant & nodered successfully.
Having updated websockets I am no longer seeing either of those nodered warnings - so thanks very much.
I can get to green lights on all areas of your flow, working price array and table, but no joy on the sensor.octopus_agile_sequence_table which appears still to not exist. I need to go and re-read your notes in case it needs to wait for cronplus and will magically appear in a few hours.
Anyway - a huge leg up on my journey to assessing solar forecast, battery soc and current load trend and making sensible charging decisions. Really brilliant!