Only spotted this by chance, and I have not seen it before. Although it looks like an Octopus API issue it may be something to do with the Node-RED http request node. The https://agileprices.co.uk/?tariff=AGILE-22-08-31®ion=L website is showing the correct figures for today (2nd) but has missing figures for yesterday (1st). My bet is on the Octopus records. Debugging stuff takes a lot of time and effort…
In the mean time, a simple patch fix can be applied to the ‘Results’ node just after the API call.
The JSONata function $distinct() returns an array removing duplicates. At the moment this for me reduces the 96 records down to 52, so it works fine, except between 16:00 and 23:00 immediately after the update when the tail end of the 96 (being the remains of today) is missing, as it was yesterday up to around 20:00.
PATCH:
In the ‘Results’ change node, set the ‘to the value’ dropdown to ‘J:’ JSONata, and add the expression
$distinct(payload.results)
This should do the trick if the problem repeats, although I suspect at 16:00 the issue will have gone away.
I have made several changes to this code (adding in BST local time and a binary ‘switch’ sensor) but every time I think it is good to release something else crops up, so I may have another think - one thing would be to pull down 144 records, strip out duplicates (flag an error if the 144 shrinks in size) then pick out the top 96 (and check the start-end dates are correct). As usual the error avoidance and checking code is considerably bigger than the code required to do the work.