Today I stumbled across an article about energy-charts.info from Fraunhofer ISE in Germany which offers data including a sustainability traffic light that should assist with better automation. I found this interesting because using this webservice various aspects of home assistant need to be addressed:
Receiving data from an restful API
Visualizing graphs with given data
Extracting data from a timebased attribute list to a sensor
Visualizing graphs with historic data outside the scope of the restful API
Creating a sensor based on forecast data
Long term sensor history topics
Automation dependent on forecast sensors
Although I don’t possess an electrical car or any other device which certainly demands for smarter control, this still might enhance my already existing automations I use with my dishwasher and washing machine. Right now they only depend on realtime information about the sun, the active PV generation and the battery level, which can easily be used in homeassistant.
Tools
- https://api.energy-charts.info/, version 1.4 on 19.12.2024
- ApexCharts card by RomRider v2.1.2 installed via HACS
Gather the data using RESTful API
First you might wanna check the API documentation at https://api.energy-charts.info or even the website https://energy-charts.info to determine if and what might be interesting to you.
They have several services - I will focus on the electricity ‘traffic signal’, but here some more:
- Public power
- Total power
- Cross border electricity trading
- Day ahead price
- Solar share
- Wind onshore share
- Wind offshore share
- and some more
In this example I want to gather the “signal”, which is the API call for the electrical traffic light “signal” which shows green for high renewable share, yellow for average renewable share and red for low renewable share or grid congestion. “Share” means the share of renewable energies in %. On the API-Page you can even call the service with the possible parameters country and postal_code.
The response will be something like:
{
"unix_seconds": [
1734649200,
1734650100,
...
],
"share": [
101.7,
103.2,
...
],
"signal": [
2,
2,
...
],
"substitute": false,
"deprecated": false
}
Now we can configure the RESTful sensor platform (RESTful - Home Assistant) - you can change the country and postal_code in the resource attribute to your needs.
Here I will load the whole data for one day into a sensor. The sensor value will just be a boolean argument which tells wether the sensor has valid data or not.
authentication: basic
# Update the data every hour
# Don't use to small numbers here to respect the free service!
scan_interval: 3600
# You'll better use ssl verification, but need to handle the certificate then
verify_ssl: false
headers:
Content-Type: application/json
User-Agent: Home Assistant
Accept-Encoding: identity
resource: https://api.energy-charts.info/signal?country=de&postal_code=86842
sensor:
- name: Electricity traffic signal
unique_id: electricity_traffic_signal
value_template: >
{% if value_json.signal is defined %}
{{ value_json.signal | default([]) | count > 0 }}
{%- else -%}
False
{%- endif -%}
json_attributes:
- unix_seconds
- share
- signal
- substitute
- deprecated
Another possibility is to create several sensors like signal_now, signal_in_one_hour, … to save just one value. Then you’ll need to extract the relevant value. Using the first column unix_seconds you can identify the index you’ll want to extract from signal or even from share. A simple example how to read one single value is
value_template: "{{ value_json['share'][-1] | float(0) }}"which will return the last value from the list (index -1 is the last item in the list which is at 23:45 just before midnight).
After having Homeassistant restarted, I can find the new sensor with state = True and all json attributes as sensor attributes.
Binary sensors
Two parts of the payload I think can be relevant on a daily use with automation - other sensors for decision making based on numerical data will come later in this post:
- ‘deprecated’ because it will tell you in advance that the service might ‘malfunction’ in your automation within the next months
- ‘substitute’ because it might tell you not to trust the data to much as they are guessed from historical data rather then taken from primary data providers.
This must be appended to your rest-block:
binary_sensor:
- name: Electricity traffic signal webservice version deprecated
unique_id: electricity_traffic_signal_webservice_version_deprecated
value_template: "{{ value_json.deprecated }}"
- name: Electricity traffic signal using historic data
unique_id: electricity_traffic_signal_using_historic_data
value_template: "{{ value_json.substitute }}"
These sensors will either be True, False or unavaliable in case the payload was invalid or service not available.
Possible errors
The sensor gets “unavailable” in case the service cannot be reached or fails somehow.
The structure might change from time to time. This example refers to the API v. 1.4. Using the value_template query I try to evaluate the payload. In case the sensor is used at different automations or widgets the sensor value can be evaluated first. The possible cause why the payload might not be valid is when the webservice has been updated changing the response. energy-chargs.info would first set the ‘deprecated’ attribute to true for I think they’ve written 6 months in advance.
Visualization
Show
My first attempt to visualize the signal using area with ApexCharts (Graph-chapter below) did not work out (yet), I want a quick solution to visualize the data using a simple table. Flex-table-card is just what I need - although without some extra tools (card_mod) it does not seem to be able to add a scrollbar and therefor is quite large.
type: custom:flex-table-card
entities:
- sensor.electricity_traffic_signal
columns:
- name: Time
data: unix_seconds
modify: new Date(x*1000).toLocaleString()
- name: Share
data: share
- name: Signal
data: signal
If you have card_mod installed you can add some code:
css:
table+: 'padding: 0px'
card_mod:
style: |
ha-card {
overflow: auto;
max-height: 200px;
}
Graph
ApexCharts can be used to visualize the data. I will show the share value as a line. The traffic light signal will be shown as colored area. This is fixed to a 2 day window and might break in case the service changes its resolution or time window - the table above can be a quick help to analyze whats going on without looking at the service payload or sensor attributes.
type: custom:apexcharts-card
experimental:
color_threshold: true
header:
show: true
title: Stromampel heute und morgen
show_states: false
colorize_states: true
now:
show: true
label: Now
graph_span: 2d
span:
start: day
apex_config:
dataLabels:
enabled: true
series:
- entity: sensor.electricity_traffic_signal
show:
legend_value: false
name: Share
yaxis_id: normal
data_generator: |
return entity.attributes.unix_seconds.map((time, index) =>
{return[new Date(time*1000).getTime(),
entity.attributes.share[index]]});
- entity: sensor.electricity_traffic_signal
name: Signal
yaxis_id: special
data_generator: |
return entity.attributes.unix_seconds.map((time, index) =>
{return[new Date(time*1000).getTime(),
entity.attributes.signal[index]]});
type: area
stroke_width: 0
opacity: 0.5
show:
legend_value: false
color_threshold:
- value: 2
color: green
- value: 1
color: yellow
- value: 0
color: red
- value: -1
color: red
yaxis:
- id: normal
max: auto
- id: special
opposite: true
max: auto
As the forecast is not always available you can split the graph in two:
graph_span: 1d
span:
start: day
graph_span: 1d
span:
start: day
offset: +1d
Above forecast shows wrong values as the API does only return values of today but not the forecast for tomorrow while it’s not available yet (the webpage states it is usually available past 7 pm, Europe/Berlin, for the next day). You can make the second graph conditional either not showing it while the forecast data is missing or just not showing the wrong values:
data_generator: |
if ( entity.attributes.unix_seconds.length > 96 ) {
return entity.attributes.unix_seconds.map((time, index) =>
{return[new Date(time*1000).getTime(),
entity.attributes.share[index]]});
} else {
return [0]
}
.... the equivalent with the signal generator.
Getting historic data
Until now the service data is just saved as a attribute list. Now the data shall be saved as sensor values for analysis of historic data.
Share of renewable energies as a sensor
I’m using a triggered template sensor - another option could be to just use an automation - to extract the nearest past share value from the list.
I thought template variables could allow me to use the same variable within availability and state, but they do not seem to be available in their context, so I need to calculate both variables twice - not a big deal. I left the code commented just in case someone knows a solution. Maybe using namespaces you can reuse the values from availability in state, but I did not checked this yet. I guess it doesn’t matter with just 96 or 192 values.
template:
trigger:
- trigger: time_pattern
# Matches every 15 minutes as this is the timeframe from energy-charts.info
minutes: /15
# Would be nice but is not available within state and availability
#variables:
# ts_now: "{{ as_timestamp(now()) | int }}"
# ts_index: "{{state_attr('sensor.electricity_traffic_signal', 'unix_seconds').index((ts_now - (ts_now % (15*60))))}}"
sensor:
- name: 'Share of renewable energies'
unique_id: share_of_renewable_energies
unit_of_measurement: '%'
availability: >-
{% set ts_now = as_timestamp(now()) | int %}
{% set ts_index = state_attr('sensor.electricity_traffic_signal', 'unix_seconds').index((ts_now - (ts_now % (15*60)))) %}
{{ ts_index|is_number and ts_index > 0 }}
state: >-
{% set ts_now = as_timestamp(now()) | int %}
{% set ts_index = state_attr('sensor.electricity_traffic_signal', 'unix_seconds').index((ts_now - (ts_now % (15*60)))) %}
{{ state_attr('sensor.electricity_traffic_signal', 'share')[ts_index] }}
Visualizing this data is easy with home assistant.
If you intend to do more (long-term) analysis you might wanna forward selected data towards a database like InfluxDB. If you do so some interesting things (and of course some work) will come up. Here a few topics that I needed to deal with or I think anyone starting might need to think about:
- Proxmox vs. standalone HAOS: I’m running HA and influxdb within Proxmox on an old NUC (i5-6500T, 32 GB RAM, several SSDs/disks for different purposes like backups). I used to have HAOS on an RaspberryPI 3 (a pain sometimes) and RPI 4. The RPI4 is quite nice to run HA and can be easily transferred to a new RPI4 in case of defective hardware (which never happened to me in several years). But still all this can be somehow limiting, why I moved on to Proxmox, running my productive and one testing HA, RaspberryMatic, InfluxDB (and more). But you should only move on into this direction if you really want to waste your spare time on topics like server maintenance, backups etc. HAOS on the RPI or similar devices is nice because it handles many tasks for you. If you have never done something with Proxmox read some articles and watch some vcasts first.
- Downsampling: I cannot find the source(s) I studied in the past and I use InfluxDB 2.x but it think Derek using InfluxDB v1 covers the theory behind it quite nicely and on InfluxData is a nice post regarding InfluxDB v2 downsampling from Anais. As long as you cannot be sure for the estimated life time of your HA- or InfluxDB-installation/experience the choice and size of data you’ll have to deal with, you don’t want to save all data at full resolution in InfluxDB. Maybe there are other solutions I’m not aware of but Downsampling is easy to implement. You’ll have your historic data in different resolutions.
- You can analyze or visualize your long-term data within InfluxDB or using e.g. Grafana or even can show graphs from InfluxDB in HA (read some thoughts and solutions in this post).
Automation based on interpreted forecast
As already mentioned I don’t have any large consumers. Just the basic ones I guess anybody reading this has and many of those are out of scope of optimizations like kitchen appliances, but others might be of interest like the dishwasher, dryer or washing machine.
Most of the time I only look at the data and decide SMART when it is possible and when its needed to do things:
- It’s just not a good idea to wait days and days forever before starting the dishwasher as some of your dishes might even get harmed by mold.
- I guess You know what I mean…
Until now I have actually only one automation use case, because my “dump” washing machine is continuing from the point at which the power supply was interrupted. I can use this for some automation:
- Automatically start the washing machine when I’m leaving my business as I’m moving towards home - that’s some self efficiency
- Automatically start the washing machine if my battery has exceeded some SOC (state of charge) also in combination with the level of PV power I’m receiving - this I’m using most of the time and quite often with my uncomplicated laundry.
- Automatically start the washing machine in the morning x hours after sunrise. Interesting for winter wonderland days.
The last one is the only one which is altruistic towards the electrical grid. And this is the only one that makes sense for me to use a forecast of renewable energies.
Minimal share of renewable energies
I’m just calculating the minimal forecast share within a time window which can be used for my decision. Another option could be to just create a “future” sensor saving these values as I’ve done it with the “Share of renewable energies”-Sensor. This sensor you could use to do several calculations with home assistants own tools. But here I will fill one extra sensor with one forecast value and I’m using an absolute minimum needed to start the machine.
template:
# This is an extension to the template defined above in the
# "Share of renewable energies as a sensor" chapter
...
- name: 'Minimal share of renewable energies next 5 hours'
unique_id: minimal_share_of_renewable_energies_next_5_hours
unit_of_measurement: '%'
availability: >-
{% set ts_now = as_timestamp(now()) | int %}
{% set ts_index = state_attr('sensor.electricity_traffic_signal', 'unix_seconds').index((ts_now - (ts_now % (15*60)))) %}
{% set hours = 5 %}
{% set frames = hours * 4 %}
{% set valid_frame_count = ts_index + frames < (state_attr('sensor.electricity_traffic_signal', 'unix_seconds')) | default([]) | count %}
{{ ts_index|is_number and ts_index > 0 and valid_frame_count }}
state: >-
{% set ns = namespace(min_share=100) %}
{% set ts_now = as_timestamp(now()) | int %}
{% set ts_index = state_attr('sensor.electricity_traffic_signal', 'unix_seconds').index((ts_now - (ts_now % (15*60)))) %}
{% set hours = 5 %}
{% set frames = hours * 4 %}
{%- for i in range(ts_index, ts_index + frames) -%}
{% set ns.min_share = min(ns.min_share, state_attr('sensor.electricity_traffic_signal', 'share')[i]) %}
{%- endfor -%}
{{ ns.min_share }}
And some automation which must be activated anytime I want it to happen:
alias: Altruistic based on renewable energy share
triggers:
- trigger: state
entity_id:
- sensor.minimal_share_of_renewable_energies_next_5_hours
from: "70"
conditions:
- alias: Washing machine is unpowered
condition: device
type: is_off
device_id: *some_device_id*
entity_id: *some_entity_id*
domain: switch
actions:
- type: turn_on
device_id: *some_device_id*
entity_id: *some_entity_id*
domain: switch
alias: Continue washing
- alias: Power off this automation
action: automation.turn_off
metadata: {}
data:
stop_actions: false
target:
entity_id: automation.*the_name_of_this_automation*
- action: notify.*my_phone*
metadata: {}
data:
message: The washing machine has started ({{trigger.entity_id}})
data:
ttl: 0
priority: high
timeout: 3600
alias: Notify me
mode: single
Using a small button I enable this automation as necessary.
This can be enhanced to start the washing machine in case the self-sufficiency is adequate. I’m just using another automation for this purpose, which also is activated using a button.