Octopus Agile - display tariff in graphs & tables, best import/export periods - all done using Node-RED and JSONata

It should probably be ‘array’ and not ‘Array’

Doh! Thanks… :slight_smile: It appears capitalised in the entities view!

I’ve been trying to get this working and im getting an error in the logs and i’m a bit stuck.

When i check the state in the developer tools, it says unavailable, however if i put a debug node on the end of the pricing flow i do see an entity with 96 attributes.

I’m quite new to this - this is my first delve into Node-Red so its quite possible im doing/have done something wrong! So any advice is much appreciated

After a quite a lot of googling, tinkering and trying stuff out, i think i have found the solution. It came down to the sensor configuration. The state class of measurement was the issue, if you clear that down and restart it the problem goes away.

Heres a pic of the config that worked for me;

Alas, my original over-exuberant use of State class ‘measurement’ hit problems last year when the rules were tightened.

Although this was picked up at the time, it was remiss of me not to try to edit my original post (I don’t think I was aware that this was actually possible at the time).

I have now re-loaded the original Node-RED flow (in first post) with slight modifications to clear the offending State Class entries.

1 Like

Hard to believe that it is one year since I first posted this project…

Although I still don’t use Octopus Agile tariff myself, I continue to look at the graphical display of agile prices, and have been tinkering with the code ever since. Last summer I attempted to add DST and local time. I waited until the time-change in October to test it, and I did for the first time in my life get up in the night just to watch the clocks change. The code did, for the most part, actually work.

From discussions on this thread, I have added a binary sensor to act as a switch. Updates to the display (graph, tables) as well as general improvements to the JSONata code itself.

The basic JSON file for the Node-RED flow is now too large to post here. I have finally got round to learning Github and all that entails, so I am now officially an old git. Yet more learning achieved.

So, the latest update is now on Github. I have collated the documentation into the read.me file, and tested the release, so time to commit.

https://github.com/oatybiscuit/Node-RED-HA-Octopus-Agile-JSONata?tab=readme-ov-file#node-red-ha-octopus-agile-jsonata

1 Like

Hi Geoff - I don’t know if you’d prefer this feedback/ask on github or here… but as I’ve already posted about a different issue there, I’m starting here with this one until you say different :slight_smile:
First of all thank you for such an amazing job in terms of intellectual effort AND such helpful writeups and support to people who’ve come unstuck, and all whilst turning down ‘buy me a coffee’ - what an amazing bloke.I am just struggling to get the entity sensor.octopus_agile_sequence_table to display anywhere* and have just discovered the following warnings repeating in nodered:

“Deprecated API warning: Calls to RED.util.evaluateJSONataExpression must include a callback. This will not be optional in Node-RED 4.0. Please identify the node from the following stack and check for an update on npm. If none is available, please notify the node author.”

18/02/2024, 18:08:41msg : string[1378]
</>
“Error: ↵ at Object.evaluateJSONataExpression (/opt/node_modules/@node-red/util/lib/util.js:775:18)↵ at JSONataService.evaluate (/config/node_modules/node-red-contrib-home-assistant-websocket/dist/common/services/JSONataService.js:58:39)↵ at TypedInputService.getValue (/config/node_modules/node-red-contrib-home-assistant-websocket/dist/common/services/TypedInputService.js:58:85)↵ at SensorController.onInput (/config/node_modules/node-red-contrib-home-assistant-websocket/dist/common/controllers/SensorBaseController.js:76:40)↵ at SensorController._InputOutputController_preOnInput (/config/node_modules/node-red-contrib-home-assistant-websocket/dist/common/controllers/InputOutputController.js:62:76)↵ at /opt/node_modules/@node-red/runtime/lib/nodes/Node.js:214:26↵ at Object.trigger (/opt/node_modules/@node-red/util/lib/hooks.js:166:13)↵ at Sensor.Node._emitInput (/opt/node_modules/@node-red/runtime/lib/nodes/Node.js:206:11)↵ at Sensor.Node.emit (/opt/node_modules/…”
</>
18/02/2024, 18:08:42msg : string[270]

Have I broken something or is there a direction of investigation you could recommend please?

*example symptom:

apexcharts-card custom card attempts produce:
Entity not available: sensor.octopus_agile_sequence_table
Entity not available: sensor.octopus_agile_sequence_table

Hi.

I have seen your messages on Github, I’m just not able to respond easily at the moment.

The issue you quote above is most likely due to out of date Websocket nodes. With Node-RED v3+ all JSONata calls now expect a callback function, which was indeed added in the HA Websocket nodes in a recent update. If you are on the latest HA core, the latest NR addon, the latest NR companion integration and the latest Websocket nodes, you should not see this error and it should all work.

I suggest checking your versions. BUT DO BE CAREFUL. There have been several breaking changes across these components, and incompatible updates may break your system.

As a start, I suggest:
Take an inventory of your versions, and ideally save a backup copy and remove your old (this) Node-RED flow completely (delete, restart HA and NR), before installing again afresh. So much has changed since my first flow that it is probably better to start again rather than try and copy over the old flow - but you will need the latest versions.

Backup first!

Thank you Geoff. I backed up this morning, and have inventoried as you kindly suggested:
HA Core 2024.1.6 vs 2024.2.2
NR Addon 17.0.7 - current
NR companion 3.1.2 vs 3.1.3
Websocket nodes 0.55.1 vs 0.63

so you were right on the money with your diagnosis. I disabled config nodes, deleted the whole flow, restarted, re installed, re configured. Whenever I try this I do run into nodered alerting me to pre-existing nodes, and I’ve not yet taught myself how to delete these from homeassistant & nodered successfully.

Having updated websockets I am no longer seeing either of those nodered warnings - so thanks very much.

I can get to green lights on all areas of your flow, working price array and table, but no joy on the sensor.octopus_agile_sequence_table which appears still to not exist. I need to go and re-read your notes in case it needs to wait for cronplus and will magically appear in a few hours.

Anyway - a huge leg up on my journey to assessing solar forecast, battery soc and current load trend and making sensible charging decisions. Really brilliant!

I would expect the sequence table to be updated only with a successful API call after 17:30,and if the raw data is/was missing from the context, no sequence.

As long as the tariff array (96 records) is there in the flow context, you can try using the ‘manual’ inject node in the flow which should trigger a recalc of the computed tables to save you waiting until tomorrow afternoon.

yes I have manual injected numerous times - I always end up with Octopus Agile Input Binary timestamp in NR being updated to now, e.g. ‘green light. false at Feb 18, 22:16’ but no context (if I understand that term correctly) for any of the nodes within the Binary Sensor bottom box of the flow (Binary Sensor - create and manage dynamic switching schedules), whereas I can see context for every node in the upper 3 boxes of the flow. When I go to HA and try to find any sign of the sequence table it seems to not exist.

Update 24 hours later. I’ve finally realised that the friendly name is what is needed to bridge the gap between the table cards and the config nodes in nodered and my config node had nothing in friendly name. Populating that with octopus_agile_sequence_table gives me a sensor entity with visible attributes. Still have a blank card apart from headings but I’m stumbling towards the light :slight_smile:

update some minutes later - now realised that in Homeassistant I am able to change entity id from format nodered_bd4485c035f519e2 to octopus_agile_sequence_table, and this (rather than friendly name) finally sorts my issue.

@NickSee

Hi Nick

I have a bit of time to look at this - do you have any issues remaining or are you now up and running OK?

I am assuming that all the issues you had were down to problems with the sensor configuration nodes.

Hi Geoff, very kind of you to offer your time like that. I am all up and running with the fruits of your work thank you. The only usability issue I have is a very minor one: on PC the Octopus Agile Rates import column is perfectly rendered in green yellow red background with black text pricing. When I use my HA app on android, the same table is rendered with white pricing text which is consequently unreadable. My fault for being on Dark Mode on my Android. Is there any property I could tweak? My googlage hasn’t lead me anywhere productive on that one yet.
I am currently working on templates to do some decision making regarding charge slots, and getting my head around the jsonata logic that produces the import windows. In my use case I would want to make use of the 3 slots my (Growatt) inverter affords me (as opposed to having one long import chunk of time coming out of Octopus Agile Period since I’ve been looking at it) and I currently manually chose the cheapest combination of slots that will get me to a target SoC by the following morning. I mention this only because I’m conscious of all the thought that you have put into your current rules and I am probably duplicating chunks of effort here. I am busy designing ways to achieve balancing consumption profiles with SoC with the output of your sensors and with the solar forecast and enjoying a LOT of learning :slight_smile:

Good to know that it is all working at last.

The agile price table uses some basic html to modify the background colour. I stick to the default light mode, but I would guess that you can modify the foreground also, or just remove that bit of the configuration.

The flow is intended to provide the basic tariff array in context store, and to return the current values for each half-hour. The extra bits are there as a guide as to what can be done, but clearly it is there to be modified for individual use. I am still looking at SOC modelling and finding it very difficult with too many variables and unreliable solar PV forecasting. The current experiment is to capture long term all solar / load figures hourly for a year, then run them through different control models and Octopus tariffs (agile, flux, fixed) to see what happens. I have decided that, if it is viable, I will go for the Flux tariff as it is more predictable and therefore potentially more manageable than Agile.

There is always a lot of learning in this - the JSONata may be rather complicated so I’m happy to assist if you are tinkering with that in particular and need any explanation or help.

Best wishes!

I’ve just started to try and have a go at this, but am struggling massively.
I have installed all the pre-reqs and configured nodered correctly I believe as it has deployed successfully however I am seeing some error . I haven’t managed to find my export API yet, but got one for the import but I’m not seeing any of the sensors in HA

Can someone help me with where I’ve gone wrong please?

In the debug, I’m getting this:
15/06/2024, 00:25:52node: SC Oct Agile Pricesmsg : error

“Error: Invalid server config”

15/06/2024, 00:25:52node: SC Oct Agile Imp Binmsg : error

“Error: Invalid server config”

15/06/2024, 00:25:52node: SC Oct Agile Seqmsg : error

“Error: Invalid server config”

15/06/2024, 00:25:52node: Octopus Agile Pricesmsg : error

“Error: Invalid entity config”

15/06/2024, 00:25:52node: Octopus Agile Import Binarymsg : error

“Error: Invalid entity config”

15/06/2024, 00:25:52node: Octopus Agile Sequencesmsg : error

“Error: Invalid entity config”

To communicate back to Home Assistant, this flow uses the WebSocket nodes (the nice blue ones). These sensor nodes each have a configuration node behind them, and all three configuration nodes are (should be) connected back to a single server node that manages the WebSocket / API connection back to HA.

Is your Node-RED as an HA addon, or stand alone?

I see from your picture that you have a small blue dot top right of two of these sensor nodes, which means that you have edited them but not deployed the flow changes. Have you followed the installation instructions, created / checked your HA Server connection, and re-attached your working server to each sensor node’s configuration node?

The instructions are long-winded, but should cover every installation from the seasoned pro to someone who has loaded Node-RED for the first time. The important point is that, as is best practice, the flow comes with the HA server configuration removed, and it is necessary (absolutely vital) to ensure that you have your own HA server working, and then to attach this server to each of the sensor nodes (to the configuration node behind the sensor node to be precise).

Please do re-check the installation guide for “Connection to Home Assistant” and the “In full detail” part as a first step.

The good news is that your API call is working, and the ‘periods left’ at 44 looks sensible. Just seems to be the WebSocket connection issue.

In Node-RED, right hand debug window, select ‘context’ tab, expand the ‘global’ section, click on the RH refresh icon.
You should see a ‘homeassistant’ context variable, as a ‘homeassistant’ object with
states and services, and isConnected and isRunning both true.

double click on each one of the Octopus Agile (Blue) boxes currently showing “error”


then click on the pencil to the right of “Entity config”:

Where it says “Add new Server”, click the drop down and choose Home Assistant.
At the very bottom left corner of this window will have “Disabled”, single click it to be “Enabled”. Then top right “Update”.

Then check the bottom left corner is still “Enabled” if not single click to “Enabled” then “Done”.

Repeat this on the other two Octopus Agile “Sequence” and “Import Binary” blue boxes.

Make sure that you have updated the Import and Export with your regional info:

Double click on “Oct Import” Then the URL should be edited for your Region:

My region product and IMPORT example: https://api.octopus.energy/v1/products/AGILE-22-08-31/electricity-tariffs/E-1R-AGILE-22-08-31-A/standard-unit-rates/?page_size=96

Thanks @Biscuit and @angusc for your help - I ended up having to delete the Home Assistant server entity and recreate it. I’ve now got the sensors working (although I have 2 of each one, one with an _2 after it) and its pretty brilliant. Thanks :slight_smile:

I have been using this for months and it has been great. Yesterday it didn’t return any data. Before I fiddle and break it :grinning:, where should I look to see what the issue might be. I tried to update it by pressing on the manual button, but still no data.

Hi @bazcurtis

If you have daily bank ups set up on your HA installation you could simply restore.