EMHASS: An Energy Management for Home Assistant

markpurcell · July 15, 2023, 10:47pm

You should keep soc_init either on this line or another line.

soc_init is another variable that tells EMHASS initial state of charge for your battery.

Part of the challenge is the curl command lines get complex with so many variables and one character out of place results in an error.

vdiogo · July 15, 2023, 11:11pm

Hey guys! Been reading this thread and the detailed documentation thoroughly to avoid raising dumb Qs, and have done some tests on my own, but I would need some help to get me moving.

My setup:

PI3 with EMHASS + Teslamate + MQTT (to listen to multiple energy sensors running on another PI4)
EMHASS is set for Cost function
4kw PV installation with 5kw inverser; two deferrable loads (EV + water resistance); no battery and no sell to the grid; single tarif
Have just created a Solcast account an created two sensors in config.yaml:
sensor.solcast_24hrs_forecast
sensor.solcast_forecast_data

Have already setup day_ahead_optimization which runs once per day (5:30am)

I’m trying to setup mpc_optim and believe i need two things now:

Add the following shell_command in config.yaml:

mpc_optim: "curl -i -H \"Content-Type: application/json\" -X POST -d '{"pv_power_forecast":{{states('sensor.solcast_24hrs_forecast')
  }}' http://localhost:5000/action/naive-mpc-optim"

Change the day_ahead_optimization from:

dayahead_optim: "curl -i -H \"Content-Type:application/json\" -X POST -d '{}' http://localhost:5000/action/dayahead-optim"

to

dayahead_optim: "curl -i -H \"Content-Type:application/json\" -X POST -d '{"pv_power_forecast":{{states('sensor.solcast_24hrs_forecast')
  }}}' http://localhost:5000/action/dayahead-optim"

rcruikshank · July 15, 2023, 11:19pm

One last error:

2023-07-16 09:04:00,047 - web_server - INFO - Setting up needed data
2023-07-16 09:04:00,049 - web_server - ERROR - ERROR: The passed data is either not a list or the length is not correct, length should be 33
2023-07-16 09:04:00,049 - web_server - ERROR - Passed type is <class 'list'> and length is 33
2023-07-16 09:04:00,051 - web_server - INFO - Retrieve hass get data method initiated...
2023-07-16 09:04:00,619 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2023-07-16 09:04:01,902 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2023-07-16 09:04:01,902 - web_server - INFO - Retrieve hass get data method initiated...
2023-07-16 09:04:02,941 - web_server - INFO -  >> Performing naive MPC optimization...
2023-07-16 09:04:02,941 - web_server - INFO - Performing naive MPC optimization
2023-07-16 09:04:02,950 - web_server - INFO - Perform an iteration of a naive MPC controller
2023-07-16 09:04:03,104 - web_server - INFO - Status: Optimal
2023-07-16 09:04:03,105 - web_server - INFO - Total value of the Cost function = -1.3
2023-07-16 09:04:03,487 - web_server - INFO - Setting up needed data
2023-07-16 09:04:03,489 - web_server - INFO -  >> Publishing data...
2023-07-16 09:04:03,489 - web_server - INFO - Publishing data to HASS instance
2023-07-16 09:04:03,508 - web_server - INFO - Successfully posted to sensor.p_pv_forecast = 443.44
2023-07-16 09:04:03,517 - web_server - INFO - Successfully posted to sensor.p_load_forecast = 740.29
2023-07-16 09:04:03,527 - web_server - INFO - Successfully posted to sensor.p_deferrable0 = 0.0
2023-07-16 09:04:03,538 - web_server - INFO - Successfully posted to sensor.p_batt_forecast = 296.85
2023-07-16 09:04:03,551 - web_server - INFO - Successfully posted to sensor.soc_batt_forecast = 13.32
2023-07-16 09:04:03,561 - web_server - INFO - Successfully posted to sensor.p_grid_forecast = 0.0
2023-07-16 09:04:03,572 - web_server - INFO - Successfully posted to sensor.total_cost_fun_value = -1.3
2023-07-16 09:04:03,583 - web_server - INFO - Successfully posted to sensor.unit_load_cost = 0.16
2023-07-16 09:04:03,591 - web_server - INFO - Successfully posted to sensor.unit_prod_price = 0.06

Shell Command:

  post_mpc_optim_solcast: "curl -i -H \"Content-Type: application/json\" -X POST -d '{\"load_cost_forecast\":{{(
        ([states('sensor.cecil_st_general_price')|float(0)] +
        state_attr('sensor.cecil_st_general_forecast', 'forecasts') |map(attribute='per_kwh')|list)[:48])
        }}, \"prod_price_forecast\":{{(
        ([states('sensor.cecil_st_feed_in_price')|float(0)] +
        state_attr('sensor.cecil_st_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list)[:48]) 
        }}, \"pv_power_forecast\":{{states('sensor.solcast_24hrs_forecast')
        }}, \"prediction_horizon\": {{min(48, (state_attr('sensor.cecil_st_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list|length)+1)
        }},\"soc_init\":{{(states('sensor.sonnenbatterie_84324_state_charge_user')|float(0))/100
        }},,\"soc_final\":0.05,\"def_total_hours\":[2]}' http://localhost:5000/action/naive-mpc-optim"

What is it refering to?

pv_power_forecast is [] due to going over API calls again. Haven’t implemented oziee yet. Thats the next task. I assume error relates to empty array but why 33?

rcruikshank · July 16, 2023, 12:09am

Error has gone now that solcast was updated at 10am

Got oziee updating at 6, 9, 12, 15 and 18 in node-red

Thanks Mark

s2719632 · July 16, 2023, 12:11am

What methods are people using to monitor their charge power in their Tesla? I have the Tesla custom integration with a polling rate of 660s. This does create some lag when calculating my sensor.power_load_no_var_loads. This does mean there are times when this sensor goes negative which will impact tomorrow’s forecast. Any thoughts on how to get a more accurate current power usage rather than waiting every 660s?

Cheers
Tom

markpurcell · July 16, 2023, 12:33am

You can increase the polling interval. I have mine set to 10 seconds.

Good practice is to have your sampling rate at least twice as fast as your control rate. So if your are running MPC every minute you should have your sensors update at least every 30 seconds.

I also have a filter on no_var_loads to ensure it has a minimum value of 0 for tomorrow’s forecast.

markpurcell · July 16, 2023, 12:42am

Sometimes your sensors will not be complete and you will get errors like this.

Ideally you would detect this before calling calling the MPC.

Unfortunately the shell_command integration doesn’t return error messages so it can be hard to detect. The rest_command integration does return these types of error so you can see if it a transient or permanent error.

s2719632 · July 16, 2023, 12:55am

Thanks again. Will this stop the car from sleeping once charging is complete? Or can you automate the polling rate time to change to 10s only while charging?

And great idea about the filter. I’ll whack that on today

markpurcell · July 16, 2023, 12:58am

Lots of ways to change polling. When driving, when charging, …

Mheloy · July 16, 2023, 1:30am

Just to give an update with the STATUS Code 500 issue I’m having… I’ve shifted the MPC call to 30second of the minute… so far so good… it seems like many REST API calls into EMHASS 00sec of every minute.

rcruikshank · July 16, 2023, 9:20am

When you see this

and this

Is this normal? What is the cause if not?

markpurcell · July 16, 2023, 9:24am

Probably just a transient error. What does the UI on port 5000 say.

These generally resolve for me after the next MPC run.

rcruikshank · July 16, 2023, 9:29am

Aligns with the data. Looks like it hasn’t progressed past midnight. MPC is running every minute:

rcruikshank · July 16, 2023, 11:28am

No change.
Had to force a ‘day ahead optimization’ and then publish via the port 5000 page.
Now I have this:

Seems to be missing an early morning charge to get over the 7am hump. I can do that manually.

markpurcell · July 16, 2023, 12:02pm

Your cost and price columns are not reflecting the Amber pricing.

You need to ensure they are included when you run an optimisation.

rcruikshank · July 16, 2023, 12:09pm

This is what is being called? Does this look right?

  post_mpc_optim_solcast: "curl -i -H \"Content-Type: application/json\" -X POST -d '{\"load_cost_forecast\":{{(
    ([states('sensor.cecil_st_general_price')|float(0)] +
    state_attr('sensor.cecil_st_general_forecast', 'forecasts') |map(attribute='per_kwh')|list)[:48])
    }}, \"prod_price_forecast\":{{(
    ([states('sensor.cecil_st_feed_in_price')|float(0)] +
    state_attr('sensor.cecil_st_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list)[:48]) 
    }}, \"pv_power_forecast\":{{([states('sensor.sonnenbatterie_84324_production_w')|int(0)] +
    state_attr('sensor.forecast_today', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',2000)|map('int')|list +
    state_attr('sensor.forecast_tomorrow', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',2000)|map('int')|list
    )| tojson
    }}, \"prediction_horizon\": {{min(48, (state_attr('sensor.cecil_st_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list|length)+1)
    }},\"soc_init\":{{(states('sensor.sonnenbatterie_84324_state_charge_user')|float(0))/100
    }},\"soc_final\":0.05,\"def_total_hours\":[2]}' http://localhost:5000/action/naive-mpc-optim"

rcruikshank · July 16, 2023, 8:59pm

No change this morning:

rcruikshank · July 16, 2023, 9:02pm

There seems to be a gap in the start time for amber data in this graph:

markpurcell · July 16, 2023, 9:06pm

Are you only calling the MPC optim and is it running correctly? There is still no Amber pricing so it looks like your MPC isn’t being called.

What do the logs say after you run the MPC optim?

You need to check your logs after it runs?

Are you running day ahead optim as well, how are you calling publish data end point?

I would suggest that you roll back to your last known working shell command and try and work out what has changed.

You can call the curl command directly from the command line for debugging, which can accelerate this process, but you may need to change some of the escape characters.

Have the logs running in one window and call the curl command expansion from developer tools in another window to determine what is going wrong.

rcruikshank · July 16, 2023, 9:10pm

Calling MPC shell command everyminute and publish_data right after it.

ok Thanks