Use Amber Electric Forecast to Optimize what hours are selected for pool pump each day

ThirtyDursty · February 10, 2022, 12:46am

Hi

Like a bit of advice or ideas about how I could use Amber Electric’s Energy price for picking the best 8hours each day to run the pool pump and heater. Essentially replicating their smart shift technology…but for whatever I want

I’m using Astral Pool’s Connect10 for the pool automation and already have some triggers that ensure it operates the heater pump when it is day time, less than a static preset price for electricity checked against the current price and the pool’s temperature (once pool is at temp, heater and pump turn off, temperature is polled off the circulation pump).

I’d like to improve on this, but figuring out how to use the Amber Price Forecast to identify the best likely hours and/or prices for a day and have that dynamically either select the hours or dynamically change the trigger and condition values.

Requirements

Pool circulation pump needs to run 8hours/day - ideally the cheapest 8hours
HeatPump and associated Pump needs to run a variable (depends on time of year and if cover was left off over night) amount of time to reach temp…ideally the cheapest hours and must be same hours as circulation pump for temperature control. maybe this variable amount of time could be calculated based on a log of the average temp gain/time over the last 7 days (how long does a 1C change take) and then set temp - current temp = Temp Change required * time for 1C change + Insurance Offset = required time to heat pool

The logic and Math make sense to me…but I’m new to programming and HA…and treating this as a learning opportunity…so just a point in the right direction and links to learning/resources would be great…but not sure where to start.

this is what the sensor.home_feed_in_forecast looks like in developer tool

state_class: measurement
forecasts: 
- duration: 30
  date: '2022-02-10'
  nem_date: '2022-02-10T11:30:00+10:00'
  per_kwh: 0.02
  spot_per_kwh: 0.02
  start_time: '2022-02-10T01:00:01+00:00'
  end_time: '2022-02-10T01:30:00+00:00'
  renewables: 96
  spike_status: none
- duration: 30
  date: '2022-02-10'
  nem_date: '2022-02-10T12:00:00+10:00'
  per_kwh: -0.03
  spot_per_kwh: -0.03
  start_time: '2022-02-10T01:30:01+00:00'
  end_time: '2022-02-10T02:00:00+00:00'
  renewables: 104
  spike_status: none
  range_min: -0.04
  range_max: 1.01
- duration: 30
  date: '2022-02-10'
  nem_date: '2022-02-10T12:30:00+10:00'
  per_kwh: -0.04
  spot_per_kwh: -0.04
  start_time: '2022-02-10T02:00:01+00:00'
  end_time: '2022-02-10T02:30:00+00:00'
  renewables: 107
  spike_status: none
  range_min: -0.04
  range_max: 0.53
- duration: 30
  date: '2022-02-10'
  nem_date: '2022-02-10T13:00:00+10:00'
  per_kwh: -0.04
  spot_per_kwh: -0.04
  start_time: '2022-02-10T02:30:01+00:00'
  end_time: '2022-02-10T03:00:00+00:00'
  renewables: 106
  spike_status: none
  range_min: -0.04
  range_max: 0.53

madpilot · February 10, 2022, 4:56am

So, a question up front: does your setup need to run for 8 hours continuously or can it be split up into 30 minute blocks?

The current SmartShift algorithm does the latter. It basically runs a planner that works out when the best time of the day will be run, and tries to stick to that unless things change too much. You will have a problem where a majority of the running will be at the end of the day, because you hit a point where you have to run the pump because there aren’t enough 30-minute slots left.

A simplier way to start with is set a threshold price “if the price is cheaper then this, run it”. DMO price is a good starting place.

You’ll need to setup a sensor that tracks how many hours the pump has run today, and a second sensor that calculates the average forecast for the remaining time (ie, already run for 2 hours? Average the next 6 hours)

If the average is less than your threshold and there is still pumping to do, keep running. If not turn off for a bit.

There is still a bit there but I believe YAML automations could do all that. Can you template from an array of attributes? Is a good starting point

ThirtyDursty · February 10, 2022, 5:41am

It does not need to run continuous. just a total of 8hours per day…could run for 30min stop for 2hours, then run again…and to be honest if it was an awful day, it could go with less or not at all for one or two days.

I’ve currently got the automation working well with a static threshold price…maybe another options is to do an average (Median, Mean or Mode, have to think of implications of each option) of the daily forecast and set the threshold as that average so it has to be below the mean forecast and then maybe set an addition failsafe of DMO+Offset or something

Just finding 1) I’ll have to keep changing my current static/manual set number through the year and 2) like today…my pool heated itself up early on 0.16c/kwh only to have a big stretch of 0.02cents mid day

ThirtyDursty · February 10, 2022, 5:51am

guess Im going to learn

madpilot · February 10, 2022, 6:16am

You could also set up a sensor that represents the mean of the eight hour period that contains the cheapest price, then use that as your threshold, so it is effectively self setting. Will be a fun problem to solve!

markpurcell · February 10, 2022, 9:24am

Nice problem, but also had application for other shiftable loads like EV charging. I.e. what are the cheapest charging windows in the next 12 hours to get my car to 80% (adding 30 kWh) with a charging rate of 10 kW - 3 hrs continuous charging. What are the 6 best 30 minute windows using Amber forecast.

I think the logic is to sort the forecast into asending order and take the lowest six windows, or in your pool car the lowest 16 values. And use that set to schedule on/ off controls. Also consider my excess solar and FIT.

My rough way of automating at the moment is based of the current pattern of low (green) Amber pricing after sunrise in the morning and high (red) pricing after sunset in the evening. My automation switches on the pool pump on green and then off on red and if I need to top up turn on again at night when the price is ok (yellow) for remaining time.

ThirtyDursty · February 13, 2022, 12:52am

So trying to tease out the attributes I want to find a mean of

Is not working yet…

but thinking the sensor needs to look something like this…although I’m very new…really not sure about the date-time set up…or any of it really

Ideally this would provide me with eh mean electricity forecast cost between NOW and 9PM…I’ll use this in conjunction with a run time timer on the pump that I’ve set up with History stats and smoother static conditions

###Amber
  - platform: template
    sensors:
      amber_daily_forcast:
        friendly_name: "Amber Forecast"
        unit_of_measurement: "°C"
        entity_id: amber.forecast
        value_template: >-
          {% set start = (datetime.datetime.utcnow) | int %}
          {% set end = 2022-02-13T17:00:01+00:00 %}
          {{ state_attr('sensor.durst_home_general_forecast', 'forecast') | selectattr('nem_date', '>=', start) | selectattr('nem_date','<=', end) | map(attribute='per_kwh') | list | mean }}

Attributes look like this…I’m in Adelaide Australia for time zone purposes.

state_class: measurement
forecasts: 
- duration: 30
  date: '2022-02-13'
  nem_date: '2022-02-13T11:30:00+10:00'
  per_kwh: -0.04
  spot_per_kwh: -0.04
  start_time: '2022-02-13T01:00:01+00:00'
  end_time: '2022-02-13T01:30:00+00:00'
  renewables: 88
  spike_status: none
- duration: 30
  date: '2022-02-13'
  nem_date: '2022-02-13T12:00:00+10:00'
  per_kwh: 0
  spot_per_kwh: 0
  start_time: '2022-02-13T01:30:01+00:00'
  end_time: '2022-02-13T02:00:00+00:00'
  renewables: 91
  spike_status: none
  range_min: -0.04
  range_max: 13.11
- duration: 30
  date: '2022-02-13'
  nem_date: '2022-02-13T12:30:00+10:00'
  per_kwh: 0
  spot_per_kwh: 0
  start_time: '2022-02-13T02:00:01+00:00'
  end_time: '2022-02-13T02:30:00+00:00'
  renewables: 92
  spike_status: none
  range_min: -0.05
  range_max: 15.1
- duration: 30
  date: '2022-02-13'
  nem_date: '2022-02-13T13:00:00+10:00'
  per_kwh: 0.01
  spot_per_kwh: 0.01
  start_time: '2022-02-13T02:30:01+00:00'
  end_time: '2022-02-13T03:00:00+00:00'
  renewables: 92
  spike_status: none
  range_min: -0.05
  range_max: 13.11

markpurcell · February 14, 2022, 7:19am

How about something like this:

"{{ state_attr('sensor.amber_general_forecast', 'forecasts') | map(attribute='per_kwh')|list}}"

amber_average_forecast: {{ state_attr('sensor.amber_general_forecast', 'forecasts') | map(attribute='per_kwh') |list |average }}

amber_general_price: {{ states('sensor.amber_general_price')}}

amber_general_price_below_average_forecast:
  value_template: "{{ states('sensor.amber_general_price')|float(0) 
                    < state_attr('sensor.amber_general_forecast', 'forecasts') | map(attribute='per_kwh') |list |average}}"

Delivers the rolling average over the next 24 hours and sensor is True if general price is below the rolling 24 hrs average, which should run the pool pump for the cheapest ~12 hours each day (but not exactly as it is the rolling average)

ThirtyDursty · February 14, 2022, 8:08am

that’s pretty cool…that energy is remarkably stable…mine was down to -0.2 and up to 0.6 now today

when I put your code into the config as a sensor for either list or list|average. I don’t get anything to display
But i have little idea still how this all works

  - platform: template
    sensors:
      amber_daily_forcast:
        friendly_name: "Amber Forecast"
        value_template:   >-
          {{ state_attr('sensor.amber_general_forecast', 'forecasts') | map(attribute='per_kwh') |list |average }}

markpurcell · February 14, 2022, 8:43am

I would be happy with -0.2, last week we got up to $15

I do find debugging yaml quite difficult as a random indentation or missing bracket can muck every thing up. What I do is drop it into Developer Tools:Template and play with it until it works, and then copy across to my sensors.yaml.

Don’t forget to change the sensor name from sensor.amber_general_forecast to what you have called it in your integration.

BrownhillCreek · February 14, 2022, 9:31am

I have similar ambitions, but it’s a bit beyond me.

Check out the Amber forum here. The API developer is very nice.

ThirtyDursty · February 14, 2022, 10:07am

Follow along here and hopefully we’ll make something useful you can cut and paste

Think we are close like to be able to eliminate high outliers. Because here in SA it is common to be super low mid day >0.1 and the. In the late afternoon evening more like 0.6 (was stung by $17/kwh the day before pulling the trigger and downloading HA to cut me and my action/reaction out of the equation) if there is a price spike of $17 it will skew the average so badly.

Key is that it waits till solar kicks in and drops the price. So if solar is projected to come on. It won’t gorge on 0.2c/kwh it will wait for 0.07 or even negative. But if it is cloudy and humid, gets it done earlier as there might not be enough hours left in the day.

Pool is hard. You could skip a day, maybe 2 with a cover. But not much more. Ideally (pool people tell me) you want 8hours of filtration and chlorinator a day. I’m not sure but is definitely working and have a perfect pool.

The rest of my house was easier with amber. Price spike turns almost everything off and end of price spike turns somethings back on. Been looking at smart breakers to take this to the next level. Just leave the fridge and computer circuits.

markpurcell · February 14, 2022, 10:09am

Sorry, I spotted my error, I needed to define as a binary_sensor:

"{{ state_attr('sensor.amber_general_forecast', 'forecasts') | map(attribute='per_kwh')|list}}"

amber_average_forecast: {{ state_attr('sensor.amber_general_forecast', 'forecasts') | map(attribute='per_kwh') |list |average }}

amber_general_price: {{ states('sensor.amber_general_price')}}

  - platform: template
    sensors:
      amber_daily_forecast:
        friendly_name: "Amber Forecast"
        value_template:   >-
          {{ state_attr('sensor.amber_general_forecast', 'forecasts') | map(attribute='per_kwh') |list |average }}

template:
  - binary_sensor:
      - name: "Amber Price Below Forecast Average"
        state:  "{{ states('sensor.amber_general_price')|float(0) < state_attr('sensor.amber_general_forecast', 'forecasts') | map(attribute='per_kwh') |list |average}}"

ThirtyDursty · February 15, 2022, 1:00am

ironed it out this morning…not sure a straight 24hour average is useful enough for me

"[0.16, 0.17, 0.18, 0.22, 0.19, 0.19, 0.18, 0.35, 0.36, 0.37, 0.33, 0.33, 0.34, 0.36, 0.36, 0.36, 0.36, 0.36, 0.36, 0.35, 0.35, 0.34, 0.34, 0.34, 0.34, 0.35, 0.35, 0.25, 0.26, 0.25, 0.24, 0.25, 0.26, 0.26]"

amber_average_forecast: 0.2958823529411765

the high evening and overnight skews against the SA solar sponge tariff that makes mid day consumption inexpensive. in this case I guess the 0.29 would work since it would run till the 0.35 hit…also worried that anytime a price spike occurs…formula would be worthless as the price spike will skew the average…

markpurcell · February 15, 2022, 1:19am

Yes, I see what you mean, as my values are pretty stable the average is close to the medium value.

So we need to calculate the median value, but there doesn’t seem to be a simple filter for that. Let me see what I can do with the sort filter and then try and grab the middle value.

ThirtyDursty · February 15, 2022, 1:57am

getting closer

{% set start = now().replace(hour=0,minute=0,second=0, microsecond=0) %}
          {% set end = (start + timedelta(days=1)) %}
            {% set start = start.strftime("%Y-%m-%dT%H:%M:%S+00:00") %}
            {% set end = end.strftime("%Y-%m-%dT%H:%M:%S+00:00") %}
"{{ state_attr('sensor.durst_home_general_forecast', 'forecasts') | selectattr('nem_date', '>=', start) | selectattr('nem_date','<=', end) | map(attribute='per_kwh') |list |average}}"

now to figure out how I adjust the end time without throwing an error

also like to ignore price spike true values

markpurcell · February 15, 2022, 2:03am

Here is the calculation for the medium value, man this yaml is bizzare (.15) was the secret sauce to split the middle value

# Amber forecasts in 30 minute intervals
{{ state_attr('sensor.amber_general_forecast', 'forecasts') | map(attribute='per_kwh')|list}}

# Sort into asending values
{{ state_attr('sensor.amber_general_forecast', 'forecasts') | map(attribute='per_kwh')|list|sort}}

#Grab 15th attribute from list which is the median value
{{ (state_attr('sensor.amber_general_forecast', 'forecasts') | map(attribute='per_kwh')|list|sort).15}}


amber_average_forecast: {{ state_attr('sensor.amber_general_forecast', 'forecasts') | map(attribute='per_kwh') |list |average }}
amber_median_forecast: {{ (state_attr('sensor.amber_general_forecast', 'forecasts') | map(attribute='per_kwh')|list|sort).15}}

amber_general_price: {{ states('sensor.amber_general_price')}}

amber_general_price_le_median:
  state: "{{ states('sensor.amber_general_price')|float(0) 
                    <= (state_attr('sensor.amber_general_forecast', 'forecasts') | map(attribute='per_kwh')|list|sort).15}}"

ThirtyDursty · February 15, 2022, 2:17am

You would think that basic statistics functions like mean, median and mode would be inherent.

ThirtyDursty · February 15, 2022, 2:29am

Actually your solution will work perfectly. But instead of finding mode. Adjust it to the number of 30min blocks you need your pool to run a day. As it will sort from low to high and pick the upper most value you require to get the job done. So 15 would be 7.5hours?

Would have thought there should be 48 values in a 24 hour forecast of 30min blocks, making 24 the median.

ThirtyDursty · February 15, 2022, 5:58am

thanks for this…I’ve used this structure to create 3 sensors

Best 3 hours (Run pump on high to turn over the pool…water chemistry and temperature mixing and stability and effective skimming…a counter will make sure this doesn’t exceed 2hours/days
Best 5 hours (Run Pool heater…also has cut outs if temp is reached)
Best 8 hours (run the pump on low for slow turnover and chlorination, acid dosing etc