Victron/Fronius off-grid PV Energy Dashboard - MQTT and modbus integration

I’ve just finished the final touches on my 100% off-grid PV installation and HA Energy Dashboard integration and just wanted to share this with the community as I’ve seen some interest on my other Victron integration thread and have noticed some questions in the forum around power vs. energy vs. utility meters and how to configure all these to integrate with the Energy Dashboard.
This is quite a long post as there is a lot to describe but I’ve tried to break it down into key sections and will publish it into 2 or 3 posts to make it more digestible.
My purpose here is to a) give back a small part of what I got from this great community and b) document what I believe is an interesting real-life case which fits right into the HA philosophy; automate as much as you can.

TLDR
100% off-grid 16kW PV installation using Victron/Fronius/Pylontech gear with KOHLER with gas backup generator exposing all system data to local network via modbus and MQTT broker ingested by hassos instance running on dedicated ESXi VM.
Main HA integrations used are modbus, MQTT, integration, derivative, utility meter, Fronius, Forecast.solar, Solcast and ESIOS API.
Lovelace UI uses custom Multiple Entity Row, Tesla style solar power card (for power flows) and Energy Dashboard (for energy accounting).
Extensive use of HA automations to independently manage loads such as irrigation system, house water pumps, and swimming pool as well as generator start/stop using extrapolation of battery discharge time (HA derivative integration) combined with next-hours and next-day solar production forecast (Solcast and Forecast.solar).
Using real-time energy cost (Spain ESIOS API - the installation is in Alicante region) we calculate the equivalent energy cost should the house be grid connected to feed the ROI - Return On Investment economic model.

Screenshot of energy dashboard on a cloudy day with generator feed-in (the grid is the generator, we are 100% off-grid; I haven’t changed the icon)

The long read

1. Intro and context

The house is 100% off-grid for energy, water and waste. Electricity is produced by a combination of Victron/Fronius/Pylontech gear with a SDMO/Kohler gas backup generator. I have 40 x 405Watt panels on the flat roof, facing due south, at a 35 degree angle, for a total maximum theoretical PV output of 16.2kW. On good days, I have seen readings of 15.8kW. These are connected in 2x2 strings of 10 panels each to a couple of Fronius Primo 8.2 solar inverters which are in turn connected to the house 230VAC single phase grid provided by the master Victron Multiplus II 48/5000 inverter/charger which drives another 2 Multiplus II 48/5000 for a total capacity of 15kW. The 3 inverters/chargers are connected to a bank of 12 Pylontech US3000C 48V Lithium batteries for a total capacity of 42kWh of storage energy. Finally, this set-up is complemented by a gas back-up generator SDMO/Kohler RESA14 11kW power which I throttle to about 80% to work on the specific gas consumption sweet spot (max electrical energy per gas consumption). The choice of multiple inverters and battery banks is deliberate to ensure resilience in case of failure of one component.
The installation is connected/monitored to/by a Victron CCGX device which exposes all the system data to the local network via modbus TCP and its inbuilt MQTT broker. The reason why both integrations are used is described later in detail.

Simplified schematic of installation

Sizing of the components, especially the battery capacity, is based on a winter daily energy consumption of ±30kWh and a summer consumption of ±60kWh the increase being the 3HP swimming pool filtration and jacuzzi electric motors. Based on historical data, battery capacity is designed to ensure generator turns on during winter season to provide up to 2% total yearly energy needs for a 98% solar coverage. We want the generator to work from time to time to ensure it is well exercised and fully operational.

2. Requirements and system architecture
Being totally isolated we produce our own electricity, water (from our own water well) and manage our waste while complying with all local regulations (specifically in terms of waste management). These are some of the key requirements for this build:

  1. No compromise on confort such as e.g. reduce electrical consumption on grey days → back-up generator
    This is our main residence and we don’t want to skip on life’s conforts. We are off-grid so once the investment is made there is no point in saving PV electrical energy. Since we opted for 100% electricity there was no way we could guarantee full supply no matter how big the battery storage capacity. The generator is a modest investment compared to batteries and is designed to run for about 100 hours maximum per year consuming roughly 300kg of propane. This means the house is not 100% “green” but neither are lithium batteries and we are sure we have backup energy no matter what.
  2. No internet dependency → 100% local data feeding the critical automations
    This one is self-explanatory.
  3. Delegation of logic to lowest level possible → Victron controller for energy, KNX for everything else
    In last resort, I don’t want to rely on a server to keep water tank full or to start the generator. For critical stuff we have 2 or 3 cascading levels of logic/automation so if the server running HA goes bust we are covered.
  4. Use of standards and vendor agnostic → Home Assistant, MQTT, modbus, etc.
    The standards part is important as I found myself down a rabbit hole more than once. When I mean standards, I include for instance running HA as hassos and not spinning up my own containers and the whole 9 yards. I know hassos is more limited than running my own containers but as much as I like to play around I want this to be as reliable and as uncomplicated as possible. One thing is development, another is production. I rely on this for production.
    Vendor tie-in is not good, limits your choices and puts you at the mercy of their plans, or lack of. This is the part where I praise the HA team for the wonderful, exceptional work they’re doing. I started out with HA 5 years ago and boy, has it come a long way. THANK YOU.

The following image is a simple architectural view of the set-up. Layer 1 implements basic logic mostly at the level of the KNX sensors/actuators (e.g. hot water production is exclusively driven by hot water tank temperature and that simple sensor can control the full process. Shading control is managed at the level of each blind, etc.).
Layer 2 represents mostly the transport layers and/or protocols used. I try to stay as close to the native capabilities of the devices and reduce interfaces.
Layer 3 implements more complex logic. KNX logic modules allow for instance calculating the average temperature in the house which in turn drives HVAC. DALI is the standard for all the lights and manages group behaviour such as dimming, colour, etc. VenusOS is the open Victron OS and manages the whole PV installation via parametrisation.
Finally level 4 implements even more complex logic with the key difference that it is not mission critical. Not to disparage HA as a tool, on the contrary, but as it must run on a computer it is more prone to failure than the lower levels. For instance calculating battery discharge rate using HA derivative I can extrapolate if it will need charging by the generator during the night so I can anticipate turning it on a couple of hours in the evening so it doesn’t disturb sleep hours. This is convenient, but not critical. If it fails, the inverter/charger at the lowest level kicks-in to turn the generator on.

3. MQTT integration

To control the PV installation I use the Victron CCGX unit but this applies to all Victron control units including the latest Cerbo. I will from now on call these the GX device as they all run Victron’s VenusOS which is openly available at https://github.com/victronenergy/venus
At one point, just for fun, I built my own device using a RPI3 with canbus header for the batteries and it worked perfectly fine. The CCGX gives me the added value of an inbuilt screen with can be convenient during set-up.
Victron’s GX devices include an in-built MQTT broker which can be activated in Menu/Settings/Services. This sets it up as a MQTT broker and sends all your installation data as MQTT message topics on the network. As mentioned above, it is possible to collect this [almost] same information using modbus over TCP with [almost] exactly the same results. MQTT is rather user-friendly in my view as using MQQT Explorer you can navigate the literally hundreds of topics available.

Note: I use Plaintext which does not require user or password and is over port 1883, but to activate it you also need to activate SSL. Using Plaintext your MQTT messages will be open to anyone on your network to read which could be considered a security risk. My take is that if anyone, a potential hacker made it that far, i.e. into your network, he/she could just as well access your GX device directly and do whatever he/she can and not go to the trouble of intercepting MQTT messages.

4. Create a keepalive automation in HA
The GX device MQTT broker needs a keepalive message every 30 or so seconds otherwise it goes to sleep. This is to ensure it doesn’t waste resources when nobody is listening. You need to have your HA send that keepalive message.
The brute force way is to send the basic keep alive message

R/<portal ID>/system/0/Serial

which makes the broker publish everything. A simple way is to create an automation such as the one below (the 12 digit hex characters topic identifier is the MAC address of your GX device which you can find in the Settings menu or in the VRM Portal if you are using it):

alias: mqtt keep alive pour casa - utilise de CCGX
description: ''
trigger:
  - platform: time_pattern
    seconds: /30
condition: []
action:
  - data:
      topic: R/04a316c4f970/system/0/Serial
    service: mqtt.publish
mode: single

From VenusOS version 2.80 Victron does not recommend doing this as there are literally hundreds of messages and you’re only going to need a couple dozen max. Nevertheless, when you are fine-tuning your system you may want to do this because that’s the convenient way to look at all the data the broker publishes while you’re making your choice. Once you figure out exactly what you need you can move to the /keepalive method described in the docs. Again I use an automation to implement it listing on the payload only the topics I use:

alias: CCGX keepalive
description: ''
trigger:
  - platform: time_pattern
    seconds: /30
condition: []
action:
  - service: mqtt.publish
    data:
      topic: R/04a316c4f970/keepalive
      payload: >-
        ["system/0/Ac/ConsumptionOnOutput/L1/Power","vebus/257/Ac/ActiveIn/L1/P","vebus/257/Dc/0/Power","system/0/Ac/PvOnOutput/L1/Power","battery/512/Soc","vebus/257/Ac/Out/L1/F","vebus/257/Ac/ActiveIn/L1/F","vebus/257/Ac/ActiveIn/Connected","battery/512/Soh","battery/512/Dc/0/Temperature"]
mode: single

This limits the chat on the line and if later I need another topic I just need to add it to the list.

5. HA MQTT integration

To integrate the GX MQTT broker into HA and consume the message topics there are basically 2 ways depending on whether you need to set-up your own MQTT broker (for instance if you use other MQTT dependent integrations), or not.
HA MQTT integration implements mosquitto_pub and mosquito_sub so you don’t need to set-up your own broker if this is your only use-case for MQTT. You just need to install HA MQTT integration and configure it to point to the IP address or your GX device (in my case 192.168.1.25) on port 1883.

If you are using your own MQTT broker, I assume here you have chosen the community add-on. If you have yet another set-up most of what follows applies equally but of course location of configuration files will be different depending on your specific installation.

Install MQTT community add-on and configure as follows:

certfile: fullchain.pem
customize:
  active: true
  folder: mosquitto
keyfile: privkey.pem
logins: []
require_certificate: false

Create a file named accesscontrollist in share/mosquitto (use the File Browser add-on, the SMB add-on or whatever you prefer) with the following line

topic readwrite #

This ensures all users can read/write on all topics. If you want to be more restrictive, there are a couple of good threads on the forum you can refer to.

Create another file in share/mosquitto named acl.conf with the following line

acl_file /share/mosquitto/accesscontrollist

This tells your MQTT broker where the access control information is.

Because you now have 2 MQTT brokers, your own and the GX one, you need to bridge them so your HA can collect the message topics published by the GX broker while connected to your own broker. Bridging means simply that your MQTT broker will act as a bridge for the GX broker, passing on the message topics you configure.
To achieve this, create file mosquitto.conf in directory share/mosquitto with the following lines:

connection casa
address 192.168.1.25:1883
topic N/04a316c4f970/# in
topic R/04a316c4f970/# out
topic W/04a316c4f970/# out

The connection statement is just the name of your choice for the broker you want to bridge. Any name will do.
The address is the IP of your GX broker and port 1883.
The topic part defines what you want to bridge in and out in plain receive mode (N), read request using mosquitto_sub (R) and write request using mosquitto_pub (W). I prefer to bridge everything because I’m already throttling the GX device by the /keepalive statement to only the topics I want.

Start the MQTT add-on and look at the logs to check everything is working as expected.

6. Topic discovery

As I mentioned, there are literally hundreds of topics published by the GX device so I use MQTT Explorer to navigate through these and select those which interest me. (here I’m connecting to the HA broker which is the same as connecting directly to the GX device as we have a bridge set-up).

Connect to your MQTT server using its IP. As we allow anonymous login we don’t need username or password. Navigate through the topics and select what you want.

When I first set-up this integration I collected just about every topic I could think of and ended up with a cluttered UI with information overload. As I pruned the topics focusing on the things I was really interested in I went for battery status and power flows and left aside the myriad voltage and current measures available in the system. As my installation is off-grid, the inverter/charger (Victron Multiplus II) uses the mini grid frequency to communicate to the PV inverters (Fronius) so I also have this topic at hand. I was also looking to automate the set-up as much as possible so looked for topics required to power the automations. In the end, I use about a dozen.

7. Power flows and energy accounting

Before moving on to the next steps I’d like to make a short diversion on the theme of power and energy. Apologies if what follows comes across as too simple or simplistic but as I read through some of the forums I noticed there is some potential confusion about power and energy and when you throw in HA’s utility meter integration and the Energy Dashboard its can get confusing.

Energy expresses the capacity to do work and is measured in Joule (SI system - other units are calories, and BTU in Imperial system). A Joule is the energy required to move a mass of one kg by 1 meter. In the context of electricity the unit commonly used is Wh (watthour) or kWh (kilowatthour). Electrical utility meters account for energy and measure kWh. Your electricity provider charges you in kWh.

Power is the rate at which work is done or energy is used/spent. The energy unit, Joule, doesn’t say how fast that mass of 1 kg is moved. If you move it fast i.e. at a fast rate, you need more power than if you move it slowly i.e. low rate. Hence power is energy/time. In the SI system the unit of power is W from James Watt. Solar panels are rated in Watt, inverters are rated in kW (or KVA in the alternative current world - not in scope here), electrical consumers such as fridges, bulbs, motors, etc. are rated in W. A 10W light bulb on for 10 hours = a 100W light bulb on for 1 hour. Both consume the same energy, cost the same money.

The following image represents the power flows in my installation. Solar panels feed the house and/or the batteries and are complemented as backup by the generator (here stopped). These are instantaneous flows (depending on the refresh rate of the system - more on that later) and are all in power - Watts. This was implemented using the excellent Tesla style power card https://github.com/reptilex/tesla-style-solar-power-card (I changed the grid icon to represent a generator).

The next image is taken from the HA Energy Dashboard and represents energy balances or accounting; it is updated on a daily basis so we are talking here about daily energy accounting. It tells me that today until now I consumed 28.7kWh of energy, more than 50% (the orange part) directly from the solar panels, the rest (green part) from the batteries. The batteries in turn were charged to 100% by the solar panels. HA updates the Energy Dashboard hourly and resets it to zero at midnight (but keeps history - more on that later).

8. Set-up of the power flows

The power flows diagram tells us what we need to measure; how much the solar panels or the generator produce, the house consumption, and the battery power which can be positive (charging) or negative (discharging). Looking through the Fronius integration and the GX device MQTT topics these are what we need:

  • From Fronius integration:
    sensor.power_ac_fronius_inverter_1 and _2 (I have 2 inverters) - PV production (this data is also available as a MQTT topic but I prefer to get it at the source)

  • From MQTT topics
    N/04a316c4f970/system/0/Ac/ConsumptionOnOutput - the house consumption
    N/04a316c4f970/vebus/257/Ac/ActiveIn/L1/P - the generator (or the grid if I was connected to the grid)
    N/04a316c4f970/vebus/257/Dc/0/Power - the power into/from the battery

These sensors are set-up in my sensor.yaml file. Depending on your installation your topics will differ but the flows will be similar.

These are raw flows and I need to calculate what power goes where. For instance, during the day and when the battery state of charge is <100%, the solar panel production goes partly to the house, partly to charge the batteries. It gets more complicated if the generator is also on, 2 inputs and 2 outputs. The Tesla card requires at minimum 5 entities, none of which is made readily available by the PV system:

        grid_to_house_entity: sensor.grid_to_house
        grid_to_battery_entity: sensor.grid_to_battery
        generation_to_battery_entity: sensor.generation_to_battery
        generation_to_house_entity: sensor.generation_to_house
        battery_to_house_entity: sensor.battery_to_house

For the sake of readability, I used the Tesla card terminology where

grid = generator

and

generation = solar panels

Here is the logic used to derive each value form the available dataset:

grid_to_house:

If the generator is on and the solar panels are producing, the system prioritises the panels to feed the house consumption. If the generator is ON, AND solar production > house consumption THEN grid_to_house = 0 (all power from generator goes to the batteries).
Otherwise, the generator contributes the missing part to the house consumption, the rest goes to the batteries. Trivial if the generator is OFF, then 0.

- sensor:
  - name: Grid to House
    unique_id: grid_to_house
    state_class: measurement
    device_class: power
    unit_of_measurement: 'W'
    state:  >
      {% if is_state('sensor.etat_generateur', '1') %}
        {% if ( states('sensor.pv_on_output') | float(0) ) > ( states('sensor.consumption_on_output') | float(0) ) %}
          0
        {% else %}
          {{ ( states('sensor.consumption_on_output') | float(0) ) - ( states('sensor.pv_on_output') | float(0) ) }}
        {% endif %}
      {% else %}
        0
      {% endif %}

grid_to_battery

This is more or less the reverse logic. If the generator is ON, AND the solar production > house consumption, THEN all the generator production goes to charge the batteries.Otherwise, it is the rest of the total production (generator + solar panels) - what is consumed by the house.

- sensor:
  - name: Grid to Battery
    unique_id: grid_to_battery
    state_class: measurement
    device_class: power
    unit_of_measurement: 'W'
    state:  >
      {% if is_state('sensor.etat_generateur', '1') %}
        {% if ( states('sensor.pv_on_output') | float(0) ) > ( states('sensor.consumption_on_output') | float(0) ) %}
          {{ states('sensor.active_in_power') | float }}
        {% else %}
          {{ ( states('sensor.active_in_power') | float(0) ) + ( states('sensor.pv_on_output') | float(0) ) - ( states('sensor.consumption_on_output') | float(0) )}}
        {% endif %}
      {% else %}
        0
      {% endif %}

generation_to_house:

We saw above that when solar and generator are producing the system prioritises the flow of power from the panels to the house. So if solar panel production > house consumption THEN what goes to the house from the solar panels = the total solar production - what is going into the battery (which is charging because solar power > house consumption) + what is going from the generator to the battery. This last term is important because if solar and generator are both producing then they can both charge the battery.

- sensor:
  - name: Generation to House
    unique_id: generation_to_house
    state_class: measurement
    device_class: power
    unit_of_measurement: 'W'
    state:  >
      {% if states('sensor.pv_on_output') | float(0) > states('sensor.consumption_on_output') | float(0) %}
        {{ states('sensor.pv_on_output') | float(0) - states('sensor.battery_power') | float(0) + states('sensor.grid_to_battery') | float(0) }}
      {% else %}
        {{ states('sensor.pv_on_output') | float(0) }}
      {% endif %}

generation_to_battery:

Here we can use entities calculated above. What goes to the battery is equal to the total solar production - what goes to the house. Nothing is gained, nothing is lost. And if there is no solar power both are zero so the difference is zero too.

- sensor:
  - name: Generation to Battery
    unique_id: generation_to_battery
    state_class: measurement
    device_class: power
    unit_of_measurement: 'W'
    state: "{{ states('sensor.pv_on_output') | float(0) - states('sensor.generation_to_house') | float(0) }}"

battery_to_house:

The last term needs to take into account the sign of the power flow into/from the battery. If positive, it is charging so no flow from battery to the house. If it is negative, all the flow is to the house as there are no other consumers. The Tesla card only accepts positive flows hence the change of sign.

- sensor:
  - name:  Battery to House
    unique_id: battery_to_house
    state_class: measurement
    device_class: power
    unit_of_measurement: 'W'
    state:  >
      {% if states('sensor.battery_power') | float(0) < 0 %}
        {{ states('sensor.battery_power') | float(0)*-1 }}
      {% else %}
        0
      {% endif %}

These are all power flows expressed in Watts and lead to the complete power flow diagram expressed in the Tesla card. I customised the grid icon, prefer to show Watts instead of kW, and added a couple of sensors to the bubble entities. For full details check-out the GitHub site. This card can be added with HACS.

      - type: custom:tesla-style-solar-power-card
        name: lapa
        show_w_not_kw: 1
        grid_icon: mdi:engine
        grid_to_house_entity: sensor.grid_to_house
        grid_to_battery_entity: sensor.grid_to_battery
        generation_to_battery_entity: sensor.generation_to_battery
        generation_to_house_entity: sensor.generation_to_house
        battery_to_house_entity: sensor.battery_to_house
        battery_extra_entity: sensor.battery_soc
        house_extra_entity: sensor.ac_out_frequency
        generation_extra_entity: sensor.solcast_forecast_today_remaining
        grid_extra_entity: sensor.ac_in_frequency
        battery_entity: sensor.battery_soc
        generation_entity: sensor.solcast_forecast_tomorrow

And another image of the end result:

9. From power to energy

In Physics Power is defined from Energy but here we’re going the other way around; we’ll calculate energy from the power flows. To do that we need to multiply power by the time it is produced, that is the integral of power over time. If power was produced or consumed steadily over time we could just multiply one by the other but in reality both change rapidly so we need a real integration function.
Below is a screen capture of a Grafana panel charting all 5 power flows over the last 12 hours; far from constant!

Enter the HA integration Integration - Riemann sum integral - Home Assistant
You’ll notice how spiky the power flows are so I use left method. The unit of prefix is k so we get kWh energy entities from the W power entities.

- platform: integration
  source: sensor.grid_to_house
  method: left
  unit_prefix: k
  name: Energy Grid to House
- platform: integration
  source: sensor.grid_to_battery
  method: left
  unit_prefix: k
  name: Energy Grid to Battery
- platform: integration
  source: sensor.generation_to_battery
  method: left
  unit_prefix: k
  name: Energy Generation to Battery
- platform: integration
  source: sensor.generation_to_house
  method: left
  unit_prefix: k
  name: Energy Generation to House
- platform: integration
  source: sensor.battery_to_house
  method: left
  unit_prefix: k
  name: Energy Battery to House
- platform: integration
  source: sensor.power_to_battery
  method: left
  unit_prefix: k
  name: Energy to Battery

Whereas the power flows go up and down, energy will always go up as it accumulates over time. Hence the following Grafana panel representing the energy spent/accumulated over the last 2 days (generator stays constant as it was not turned on during this period).

Remark: we’re in winter so as the number of sunny hours is less than the number of night hours the house gets most of its energy from the batteries, which are then charged during the day. In the power flows panel above you’ll notice the large peak of power from the solar panels to the batteries during the sun hours.

These energy accumulated graphs are interesting to look at long term trends but I’m interested in daily and monthly periods. Daily because that is our natural rhythm, and monthly because that is the invoicing frequency and even if I don’t pay invoices I’d like to know how much my investment is worth.

So we’ve started with the power flows in W which are instantaneous, integrated them over time to get energy in kWh, and now want to measure energy in daily and monthly buckets. Enter the HA Energy Dashboard and Utility Meters.

10. Energy Dashboard

HA Energy Dashboard is a convenient and good looking way to convert your power flows into daily energy balances, with the added bonus of cost calculation.

You’ll recognise the different energy entities we defined until now including the 2 Fronius PV inverters (it is night now, the solar inverters turn off on their own so those sensors are unavailable - all normal stuff).
HA Energy Dashboard accumulates these on a daily basis, weekly, monthly and yearly so we got our cycles covered right? Well, mostly…

The entities used by Energy Dashboard are internal to HA and as far as I know not available to be consumed directly, only through the dashboard. But we can easily create them using the Utility Meter integration Utility Meter - Home Assistant

We just need to configure as many entities as we want, each one for a cycle (hourly, daily, weekly, etc.) from the raw energy entities, and we get counters which reset every period. Here are some I have configured:

# utility meters
utility_meter:
  daily_energy_generation_to_battery:
    source: sensor.energy_generation_to_battery
    name: Daily Energy Generation to Battery
    cycle: daily
  daily_energy_generation_to_house:
    source: sensor.energy_generation_to_house
    name: Daily Energy Generation to House
    cycle: daily
  daily_energy_battery_to_house:
    source: sensor.energy_battery_to_house
    name: Daily Energy Battery to House
    cycle: daily

… and their graphical representation:

This doesn’t carry any more information than the Energy Dashboard, just different format and entities I can directly manipulate. Which I need to in order to get to the cost part of the picture.

… end of this post … to be followed

12 Likes

… second post …

11. Calculating energy costs

When you configure the Energy Dashboard you can track costs for the Grid energy, both incoming and out coming.

But this doesn’t work for me on 2 counts; first, I’m not connected to the grid, and secondly I want to track costs based on the hourly rate of the Consumer Price in Spain also known as PVPC published by the Energy entity ESIOS. I started out by programming the REST call myself until I came across the excellent HACS integration Spain electricity hourly pricing (PVPC) - Home Assistant and its associated Lovelace card https://github.com/danimart1991/pvpc-hourly-pricing-card

Utility Meter allows you to define tariffs and you get as many entities as there are tariffs. You can then use those entities which accumulate the kWh energy consumed at that tariff and multiply them by the respective tariff to get the cost. You change tariff with an automation; say for instance change to next tariff every hour. But with hourly pricing that is 24 tariffs and the whole thing just becomes too complicated. I tried using the input_number type solution to persist the consumed energy value for the hour and then multiply by the tariff, adding it up all the way during the day but it was clunky.

After many trials and errors I settled for what I believe is a simple and elegant enough solution which totally suits my needs. I use an automation which triggers when the entity state changes, and use the trigger.from_state.state and trigger.to_state.state to calculate the amount of energy consumed in that short period, and multiply by the applicable tariff. This way I don’t need to persist any intermediate values, HA does it for me, and I don’t need to worry about changing the tariff, it’s done on its own.

alias: Calcul cout énergie panneaux a maison
description: ''
trigger:
  - platform: state
    entity_id: sensor.daily_energy_generation_to_house
condition: []
action:
  - service: input_number.set_value
    target:
      entity_id: input_number.coste_energia_paneles_a_casa
    data:
      value: >-
        {{ states('input_number.coste_energia_paneles_a_casa') | float(0) +
        (trigger.to_state.state | float(0) - trigger.from_state.state |
        float(0)) * states('sensor.esios_pvpc') | float(0) }}
mode: single

I run an automation for each energy component which, were I connected to the grid, would be invoiced by the electric company, that is solar panel and generator production. What the battery feeds into the house has already been accounted for as the battery is charged by the solar panels.

Or in graphic form (PVPC scale 10 so we can see meaningful variation).

To the total energy cost I have to add the base power fee as well as taxes which would add-up to roughly 10€ for the day. Sunday’s tariffs are quite low, on a normal weekday it goes up to 25€ and more. The reason I do this is, apart from the feel good factor of not having to pay that (but I did pay the investment of course) to calculate a statistical ROI period. When I designed the installation 3 years ago equipment costs were much higher and electricity costs much lower. My calculated payback period was less than 10 years. With the real installation costs being much lower and electricity much higher I project about 4 years payback time.

12. Conclusion and next steps

I covered the integration of the Fronius/Victron/Pylontech/Kohler off-grid PV installation using standard components and integrations. Where non-core stuff is used I revert to HACS.

After a short digression on energy and power I went through the different steps to go from the available data to the power flows, energy entities allowing configuration of the Energy Dashboard, Utility Meters to provide accessible entities I can manipulate at will, and the tariffs/cost calculation in this specific case which is not covered by the Energy Dashboard.

The cost calculation solution is quite simple and I believe elegant but introduces a potencial issue; as HA ingests the MQTT topics from the GX device I cannot control the update frequency of these which is in the order of the second. As the cost automations are triggered by the state change of these entities I get a lot of automation triggers and recorder data with frankly, I don’t need. What can I do with sub-second cost updates? I’d settle for a much longer update frequency, which would lower the load (no problem there, running on a powerful machine but why waste resources?) and bloated recorder database (using MariaDB). None of these issues is critical but all in all don’t satisfy me as a globally optimised solution so I looked for alternatives and found one in the shape of modbus.

The GX devices can also expose system data over modbus TCP and using that, I can control entity update times and throttle the whole system to my liking, while still using exactly the same base solutions. That will be for the next post as it introduces a complication but also opens up new opportunities not available with the GX MQTT broker.

I’ll publish the next post within a few days.

7 Likes

That write down is so impressiv, thank you for sharing your setup and im really jellous🤗

1 Like

Thank you for the detailed inflammations :+1: :pray: :smiling_face_with_three_hearts:

1 Like

Are you using your own MQQT broker, particularly the Home Assistant add on?

Suddenly running again. I use mosquito as add on of hassio.
Using samba share to access and create the conf file for mosquito as before.
Played with the sensor definitions and suddenly it was running again.

Only the keep alive is not working.
Actually I use auto reload on chrome to the victtron portal to keep it running.

Anything wrong with the

Happy it’s working although I’d try to understand why it stopped doing so, it’s never good to rely on happenstance.
As for the keep alive automation can you paste the complete code? Difficult to say anything without the whole context.

I found something strange today.
When the Victron is running on Firmware 2.81 everything is OK.
After changing to the actual 2.84 its not longer working :frowning:

Does someone know if they have changed something according the mqtt ?

Was there a particular reason you upgraded to 2.84? The reason I updated to 2.80 was to have access to the /keepalive option. In general I prefer not to update until I get everything to work 100% and I know why it’s working. So I’d go back to the last known working configuration and start from there.
I understand you haven’t been able to get the keepalive automation to work. Maybe that’s a next step. And also to fully understand the reason why you lost those sensors a while ago.
For the keepalive automation if you want help please post the full automation code. Use the formatting option </> above to cleanly format the code.

The update was set to autoupdate. I changed it to off.
The keep alive has the same format as in your example…
R/04a316c4f970/system/0/Serial

My yaml…

alias: Victron keep alive
description: ‘’
trigger:

  • platform: time_pattern
    seconds: ‘10’
    condition: []
    action:
  • service: mqtt.publish
    data:
    topic: R/d41243b612e6/system/0/Serial
    mode: single

Any idea?

Screenshot…

I believe the problem is with the time format. ‘10’ means every minute on the 10 seconds whereas what you need is /10 or /30 which means every 10 or 30 seconds.
With what you have now you will have bursts of data every minute from seconds 10 to 40 (the Cerbo MQTT stops sending after 30 seconds) and silence from seconds 41 to 9 of the next minute.
Try with /10 or anything less than /30 and let me know if it works.

You are right. It was every xx:10.
I changed it to /20 ad will check tomorrow if it’s constant.

Thanks

1 Like

Up to now running :sunglasses:

1 Like

First of all, thank you very much for this extremely interesting article. If I may, I would like to ask you a few questions about your PV installation. I know this is not really the purpose of this thread, but I don’t know where to ask.
Hopefully, construction on my new house should start soon, so I’m trying to figure out my photovoltaic setup. My setup will be slightly different than yours because instead of being connected to a generator, I will be connected to the grid. I am new to the Victron world and have some questions about your configuration. If I am not mistaken, in Victron terminology, your system is called an ESS. ES systems are very flexible as to where you connect your PV array. It can be a combination of: “PV to AC input” and/or “PV to AC output” and/or “PV to DC”.

In your case you are connecting the Fronius PV inverters on the “AC side” of the MultiPlus II inverters/chargers and the batteries on the “DC side”. So, if your PV production is bigger than the consumption the energy from the PV goes directly to the house’s load and the left-over goes to the battery through the MultiPlus (used as a charger). If the house’s consumption is bigger than the PV production then the MultiPlus (used as an inverter) is used to help the Fronius by taking energy from the battery.
First question: to use the energy from the battery, the Victron AC output must be connected to the house loads, but to charge the battery, the Victron AC input must be connected to the Fronius and thus also to the house loads. But does this mean that the AC input and AC output of the Victron must be connected together? Is this what you have done?
But my main question is: why did you choose to use some Fronius inverter placed on the “AC input” rather than use Victron MPPT used on the “DC side”. It seems that Victron and Fronius have been working closely together on this solution but I have hard time to see the benefit of one solution over the other. Personally, I am considering the following configuration:

In this configuration you have a DC bus that connects

  • as many MPPT as needed to take input from the PV arrays
  • as many MultiPlus as needed to provide the necessary power to the house
  • as many batteries as needed to store the energy

So, for your installation the two Fronius would be replaced by 4 MPPT 150/70 each taking a string of 10 panels (in parallel 2 strings of 5 panels in series) and three (or two) Victron 48/8000/110 (or Victron 48/5000/70)

Do you think my solution will work and do you know if one solution is better than the other?

1 Like

Actually, I think I can already reply to my first question. Your schematic is confusing on the subject of AC connections as it does not separate the AC in and AC out on the MultiPlus II.
In practice you probably connect the generator and the two Fronius outputs to the 3 Multiplus AC ins and the 3 AC outs of the Victron are connected to the house loads?

Hi there.

First on the type of installation. There are basically 2 types, AC-coupled and DC-coupled. Mine is AC-coupled, the one you propose below is DC-coupled (actually I have a second installation separate from the main one in the well-house which drives the DC pump motor and that one is DC-coupled for reasons I’ll explain below).

If your loads are mostly AC, such as in a normal house, then it’s better to go for an AC-coupled model as mine. This means the Fronius grid-inverters produce AC power which is consumed directly without having to go through the batteries. When the AC loads are higher than what the Fronius grid-inverters can produce then indeed the Multiplus kick-in as inverters using energy from the batteries. But as in most typical private houses my consumption is higher during the day when the Fronius can produce AC power which is directly consumed.

If your loads are mostly DC then you’re better off with a DC-coupled model as I use in my well-house because the pump motor is DC.

The reason for these choices is mostly efficiency. If your loads are AC and your higher consumption is during the day then by consuming directly from the Fronius grid-connected inverters you are saving yourself the losses of MPPT DC to battery, battery to Multiplus inverter AC. In other words, you are saving a trip of the energy through the batteries which is good for efficiency and also for the batteries as they are less stressed. I hope this is clear enough.

Secondly, you’d notice I’m writing “Fronius grid-connected inverters”, that’s for a reason. And this goes to your ESS discussion. My installation is not ESS, it is off-grid or mini-grid depending on the manufacturer lingo. For Fronius it is mini-grid 50 as I’m using 50Hz. For typical AC inverters like the Fronius to work they need to be connected to an existing grid, that’s why they’re called “grid-connected inverters”. On their own, they do nothing, they need a grid to connect to and depending on whether you are off-grid like me, or public grid-connected in whichever country, the first thing you need to do in setting-up these inverters is to configure what type of grid and which country you’re in, that is because of the need to comply with local regulations like “can you feed back to the grid?” etc.

To recap, my installation is off-grid or mini-grid which mens I’m totally isolated like in a boat on the ocean. Your installation as you explain it is the ESS type as defined by Victron where you are connected to a public grid and you either consume what you produce, store it in your batteries for later use, or feed it back to the grid if you’re allowed.

In my off-grid case the Fronius still need to connect to an existing grid and that is provided by the Multiplus sourcing their energy from the batteries. So my start-up sequence is:

  • 1 get battery banks on

  • 2 start Multiplus which then create a 230V50Hz grid on the AC output

  • 3 start the Fronius inverters which can the connect to the existing mini-grid

Important note: in an off-grid AC-coupled installation if your batteries run too low and disconnect you’re in real trouble as the only way to get things back on is to charge the batteries to a level where they come back on-line. But as the batteries are off, your Multiplus cannot create a grid, and the Fronius cannot start-up as they have no grid to connect to. You then need to charge the batteries through an external source like a generator. My generator always kicks-in well before the batteries are too low but should that happen that sequence is the only way to get things back on if the batteries should go too low. To avoid this awkward situation, it is recommended to set aside a few panels and connect them DC-coupled through a MPPT type charge controller. In this case, should the batteries go too low and disconnect, the MPPTs will charge them when there is sun and once their charge is above the cut-off level the whole installation will come back on-line. That is good practice but not absolutely necessary in AC-coupled installations. DC-coupled installations are by design immune from this problem as if the batteries go too low then the MPPTs will just charge them back when there is sun.

Now about where to connect the Fronius. The Fronius are connected to AC-output as that is where there Multiplus generated grid is. The Multiplus have the “intelligence” to manage the three things on the AC-output; their own created grid, the consumer loads, and the Fronius production. If the Fronius production meets the loads requirements then they feed directly the loads from the Fronius. If not, then they compensate by adding energy from the batteries. If the Fronius production capacity is higher than the loads and the battery state of charge SOC is <100% than the Multiplus use the excess production to charge the batteries (actually converting back the AC to DC). If the Fronius production capacity exceeds the loads and the batteries are 100% then the Multiplus tells the Fronius to modulate their production, they do this by increasing the grid frequency up to 52Hz to throttle and eventually shut-off the Fronius production. As a side note, this frequency variation means that all AC motors running in my house, including the pool pump, accelerate when the batteries are charged, a very noticeable effect. And it also means that home appliances with digital watches like ovens go beserk because in most implementations the manufacturer didn’t care about using a 10 cents oscillator but relies on the grid frequency as the time beat reference (I speak of experience as my Bosch oven’s digital watches are useless, bad bad implementation Bosch!).

The AC-input is reserved to either a public grid like in your case or a generator like in my case. The difference between a Quattro and a Multiplus is that Quattros have 2 AC-inputs, one for grid and one for generator. They are useful in boats for instance where when you are at sea you have a generator, and when at port a grid connection. In your diagram there is a PV inverter on the AC input and that is because you have a public grid to connect to and that is the one that drives the whole system. When you are connected to a public grid you need to “obey” its rules which are basically tension, typically 230V, frequency, 50 or 60 Hz and to be in-step with that frequency. When connected to a public grid both the Fronius and the Multiplus take the public grid parameters and adapt to them, they sync to them before coming on-line. In my case, off-grid, I have no public grid so it is the Multiplus that creates the mini-grid to which the Fronius sync to (but here I’m repeating myself).

Long note but to recap:

  • My installation is AC-coupled because my loads are all AC and I consume most energy during the day, when the sun is out, directly from the Fronius saving a battery to DC-AC inverter trajectory

  • My installation is off-grid or mini-grid not ESS. ESS is when you are connected to public grid

  • My Fronius are connected at AC-output because that is where the mini-grid created by the Multiplus is

  • My generator is connected at AC-input but that could as well be a public grid (and then I’d be ESS)

Recommendation? If your loads are mostly AC then go for AC-coupled with a couple of panels DC-coupled to avoid the possible bock-out situation. That’s the most efficient solution.

Hope this is reasonably clear and if not, shoot.

2 Likes

Thank you very much. Your explanation is very clear and full of very useful information and tips.

After watching several videos and articles on the subject and with your detailed information, I think I now have a better understanding of the pros and cons of each solution. One thing I didn’t originally understand is that even though the Fronius inverter is on the “AC-out” of the MultiPlus, it can actually send power back to the battery and even to the grid. The term “AC-out” on the MultiPlus II is somewhat confusing because it is actually a “bi-directional” connection.

From what I have seen it seems that in most cases it is better to connect the Fronius on the “output side” of the MultiPlus for two reasons:

  • If the Fronius is on the “input side” then in case of grid outage the Fronius stop working
  • If the “input side” only received the AC from the grid then you do not need to put a meter to ensure that no power is send back to the grid (the MultiPlus take care of it automatically)

So in my case using Fronius for the PV would look like this

And potentially a mixed solution would look like this

Again thanks for spending time to explain :sunglasses:

1 Like

For information there is a very interesting webinar (mars 2022) from Fronius and Victron that covers most of the topics that we just discussed for off-grid system. For people in the process of defining their system, I would highly recommend watching it at Advanced Webinar: AC Coupling with Fronius & Victron (AFR) - YouTube
They explain the advantages and drawbacks of AC versus DC coupling and Fronius AC-in versus AC-out connection. They also explain the 1:1 rule and now I think I understand why you have two 8.2 kW Fronius inverters and three 5 kW MultiPlus II inverters/chargers

From what I understand, Fronius has a Datamanager system with a portal access that allows to see all the parameters of the Fronius inverters similar to the VRM portal. It seems that they also communicate most (all) information to the Victron GX system through WiFi or Modbus TCP.
I am curious to know if you have tried to “integrate” the Fronius information into Home Assistant or if you are reading Fronius information only through the Victron MQTT server?

I think I will start with a system using AC-output coupled system with a Fronius 8.2 kW connected to a MultiPlus II 8000 W to follow the 1:1 rule. For PV I will start 20 * 400 W panels.

If I need more power I can add 20 more PV panels and two MPPT 150/75 connected on the DC side. I like this configuration that combines best of both world and allow a simple upgrade without forcing to add immediately a second Victron inverter to follow the 1:1 rule (I just learned that as of today it is not yet possible to put two 8 kW Victron in parallel—should be fixed)

1 Like

The Fronius integrate directly with the Victron CCGX (or whatever Venus device you’re using) via Ethernet. I guess you can also use WiFi but I prefer cabled connections whenever possible. Since I have 2 Fronius, one is the master and the other the slave. Venus recognises the master as you will see in the screen capture below but will take both into account in the whole system setup.

Once the Fronius inverters are detected you can parametrize a couple of things such as where it is connected and whether you want it to show up on the main screen

The end result looks like this in the device list menu

and in the pages screen

This Fronius/Victron integration is quite complete and you do get a lot of Fronius data through the Venus bus or MQTT interface. But if you’re using HA you can directly install the Fronius integration which will define a lot of entities (in my case 34 in total) which give me all the data I need without having to go through yet another portal (the Fronius one).

This is a sample of the entities defined by the Fronius integration in HA

About your system I’m glad you now understand the ins and outs of the 2 types of coupling and can make an informed decision.

1 Like