Victron VRM Portal API data integration

Sorry for the late response. I’m not using REST sensor anymore because of what you say, need to change it frequently. I’m using the MQTT server built into the CCGX (or any other Victron VenusOS device) and HA MQTT broker to read the values I want. The solution is explained above in another post. The CCGX publishes lots of data so you may want to analyze and select what you want by using MQTT Explorer. If you have trouble setting this up fire away your questions.

1 Like

Does this work if both installations are remote? I guess I need to open ports in both routers and maybe its way to complex :S

Do you mean your Victron installation is not on the same site as your Home Assistant instance? If that is the case take a look at Victron’s GitHub page GitHub - victronenergy/dbus-mqtt: Venus OS service mapping the D-Bus on Venus OS to MQTT
The mqtt broker on your CCGX local device forwards all communication to the central Victron mqtt broker “in the cloud” from which you can recover the messages using your credentials. No need to open ports or other stuff.
Give it a try and let us know if it worked for you.

1 Like

For those of you with remote (not local) integration from victron. You need to go with JSON from the VRM API as explained above by @lsgv
The templating I found at the beginning of this post BY the ID was dropping every now and then when victron changed the ordered suddenly, I see now that the templating was later improved but didn’t see that at the time.

Here is my templating and it is working:

OLD:

      xxxx_fridge_temp_old:
        friendly_name: "Temp. Nevera old"
        icon_template: mdi:thermometer
        unit_of_measurement: "ºC"
        value_template: '{{ states.sensor.vrm_data.attributes.records[177]["formattedValue"][:-2] | float () }}'

NEW:

      xxxx_fridge_temp:
        friendly_name: "Temp. Nevera"
        icon_template: mdi:thermometer
        unit_of_measurement: "ºC"
        value_template: >-
          {% set il = state_attr('sensor.vrm_data', 'records') %}
          {% for result in il %}
            {% if result['instance'] == 26 and result['description'] == "Temperature" %}
              {{ result['formattedValue'].split(" ")[0]|float(-99) }}
            {% endif %}
          {% endfor %}

1 Like

This looks very promising, defeneatly going to give it a try if the REST fails again with new numbers from victron.

Hi, Michael from Germany.

Is there anywhere a step by step tutorial how to integrate my Cerbo GX into Hassio?

Thanks

Michael

Sure, happy to help. I’ve got so much out of this forum, glad to give something back.

The following guide addresses installations where a) your Cerbo or any other Victor GX device is on the same network as the HA server, and b) you are NOT using MQTT for anything else within the HA setup including NodeRed. If you are using MQTT in HA for other things then I’ll write up another section on that, just let me know.

1. Connect to your Cerbo
Best is to connect locally using the Cerbo local IP on your browser. Mine is 192.168.1.25. Go to Menu/Settings/Services and enable MQTT on LAN (SSL) and (Plaintext).
This sets up the Cerbo as a MQTT broker and sends all your installation data as MQTT message topics on the network.

Note: I use Plaintext which does not require user or password and is over port 1883, but to activate it you also need to activate SSL. Using Plaintext your MQTT messages will be open to anyone on your network to read which could be considered a security risk. My take is that if anyone, a potential hacker made it that far, i.e. into your network, he/she could just as well access your Cerbo directly and do whatever he/she can and not go to the trouble of intercepting MQTT messages.

2. In your HA install MQTT integration
and configure it to connect to broker [your.cerbo.localip.here] on port 1883. No username or password are needed. Save.

This connects your HA to the Cerbo MQTT broker and allows it to read (or write) messages from/to it.

3. Create a keepalive automation in HA
The Cerbo MQTT broker needs a keepalive message every 30 so seconds otherwise it goes to sleep. This is to ensure it doesn’t waste CERBO resources when nobody is listening. You need to have your HA send that keepalive message. The simplest way I found is to wrote an automation such as below.
This runs every 30 seconds, and publishes a simple message to the Cerbo broker. If you don’t send the keepalive message you won’t get anything back from the Cerbo broker, so keep this automation running all the time.

alias: mqtt keep alive pour casa - utilise de CCGX
description: ''
trigger:
  - platform: time_pattern
    seconds: /30
condition: []
action:
  - data:
      topic: R/yourCerboMACaddresshere/system/0/Serial
    service: mqtt.publish
mode: single

IMPORTANT! The broker address is your Cerbo MAC address which you already know if you connected to the VRM portal, or you can find in your Cerbo menu under Settings/VRM online portal/VRM Portal ID - it is a sequence od 12 hex characters

4. Install MQTT Explorer or similar package on your computer
(the one you’re using, not the HA server) and logon to the Cerbo MQTT broker.

IF the keepalive is running, you will see a hierarchy of messages depending on your installation, which devices you have, etc. It’s all very self-explanatory so just browse through it and identify the data you want/need. For instance here I have the battery data. current, power, etc.

A WORD OF ADVICE: You’ll probably be tempted to collect as much data as you can just because, you can, and please indulge yourself as I did but over time I figured I really didn’t need to know so much and stuck to the really important things like battery SOC, solar yield, and a few others. The main question is "what are you going to do with it?"
If it is to just look at the data, then fine, but the really important one is the data you need to carry on some automation or logic. For instance, I use the battery SOC and take the derivative (HA integration) to extrapolate when the batteries’ SOC will drop to 20% so I can anticipate turning-on the generator (my house is 100% off-grid) IF the next day solar yield forecast (Solcast integration) is below what I need. That is really useful data in my case.

5. In HA add the MQTT sensors
you want to your configuration. Either in the configuration.yaml file or if you are breaking up your configuration in for instance a sensor.yaml file (and don’t forget the include statement in the configuration.yaml file).

Below my example of battery SOC (in MQTT Explorer once you get to a topic you want, just select the copy icon - check out the red arrow on the picture above - so you don’t make mistakes typing etc.)

# Batterie SOC
- platform: mqtt
  name: Battery SOC
  unique_id: battery_soc
  state_topic: 'N/yourCerboMACaddresshere/battery/512/Soc'
  value_template: "{{ value_json.value | float(0) | round(2) }}"
  unit_of_measurement: '%'

6. Add the sensors to your Lovelace, automations or whatever, and enjoy!

PS: If you are interested in using all this in the Energy panel of HA, just let me know and I’ll write another post on that. It took me some time to get to grips with it and I’ve seen a lot of discussions on the forum about power and energy which makes me think there is quite some confusion.

5 Likes

Thanks for the detailed explanation.
Yes, I’m interested in using this info in the energy monitor panel.

1 Like

Thank you very much for the explanation.
I managed to get some values for testing.

Is there a complete list of topics with some explanation available?
I like to switch on my aircon in summer if e. G.

  • soc >90%
  • ac pv power > 1000w
  • grid l1+l2+l3 > 100w

My multiplus 2 configuration looks like

And I don’t like to send energy back to the grid.

1 Like

Hi. Not that I know. The topics are mostly self-explanatory but it is best to navigate through them and maybe double check values with what you see on your VRM dashboard to confirm it is what you need.
The topics depend on your specific set-up but usually there is a System topic which collects most of the information you would need. MQTT Explorer has a search field on the top which you can also use.
Let me know if you make any progress.

Not longer working since 01.02 2022.
Trying different solutions, no luck.
Actual cerbo gx fw 2.8.1
MQTT Explore gets all values but hassio not getting any values :frowning:

I just upgraded directly to 2022.2.3 with no issue. Unless there is some new functionality I want to test in the new release I usually wait 1 or 2 fixes before upgrading; I’ve noticed it is more stable that way.
Back to you; are you using HA MQTT broker? Anything in the logs? Have you restarted the MQTT integration? And tried to listen to a topic directly from that integration?

If you’re still interested I’ve just posted a guide including the setup of the Energy Dashboard. You’ll find it under Victron/Fronius off-grid PV Energy Dashboard - MQTT and modbus integration

2 Likes

Sorry for the late reply. Mqtt explorer is working fine. But subscripe to listen in mosquitto gives no result. I’ve checked the mosquito conf. Still like before. I deleted all sensors but one. No change. Tried " and ’ in the sensor definition… No success.
I don’t know where to look for :roll_eyes:

As mentioned above I wrote a detailed post on this integration; Victron/Fronius off-grid PV Energy Dashboard - MQTT and modbus integration
Try to go through steps 3 to 6; on point 6 point your MQTT explorer to your Home Assistant IP to check if the connection is working.
Let me know the results.

2 Likes

@lsgv Thank you so much for your work!
I’ve been lurking like 3 years thinking: “There will soon be a Victron Integration - as popular as it is worldwide. Even more so since it is dutch and HA is strong there”.
Now I bit the bullet and finally got it to work!

I just struggle a bit with the value template to extract some information - how e.g. would I get the valueEnum Text of the charge state?

{
	"idSite": 11111,
	"timestamp": 1646491465,
	"Device": "Solar Charger",
	"instance": 288,
	"idDataAttribute": 85,
	"description": "Charge state",
	"formatWithUnit": "%s",
	"dbusServiceType": "solarcharger",
	"dbusPath": "/State",
	"code": "ScS",
	"bitmask": 0,
	"formattedValue": "Float",
	"rawValue": 5,
	"dataAttributeEnumValues": [
		{
			"nameEnum": "Off",
			"valueEnum": 0
		},
		{
			"nameEnum": "Fault",
			"valueEnum": 2
		},
		{
			"nameEnum": "Bulk",
			"valueEnum": 3
		},
		{
			"nameEnum": "Absorption",
			"valueEnum": 4
		},
		{
			"nameEnum": "Float",
			"valueEnum": 5
		},
		{
			"nameEnum": "Storage",
			"valueEnum": 6
		},
		{
			"nameEnum": "Equalize",
			"valueEnum": 7
		},
		{
			"nameEnum": "Off",
			"valueEnum": 245
		},
		{
			"nameEnum": "Equalize",
			"valueEnum": 247
		},
		{
			"nameEnum": "Ext. Control",
			"valueEnum": 252
		}
	],
	"id": 98
},

In the Victron world there are so many system configuration variants and options that I believe it would be close to impossible to create an HA integration that would satisfy everyone. It’s magic that Victron publishes so much and gives everyone access to the inner workings of their system. And we can build solid bespoke integrations using that.

In your case, I believe you are still using the REST mode, querying the cloud VRM services. I started that way too but soon found out it is suboptimal. Especially when you have all the information served on the local network both by the MQTT broker and modbus.

I recommend not getting hung up on extracting values from JSON responses as the structure of these will change over time and your carefully tuned templates will stop working. Go the MQTT or the modbus route depending on whichever you prefer or are more comfortable with. More user friendly, more robust.

A couple of posts above in this thread you’ll see a link to a more detailed guide using MQTT including the creation of power, energy, and utility counter sensors and integration with HA Energy Dashboard. Why don’t you take a look and see if that works for you, and come back with any questions you may have?

Yeah - I am sure you’re right and appreciate the recommendation to do it locally.
It’s a bit offtopic though: I use Victron stuff in my campervan. Last summer I played around with a nice Teltonika RUTX11 Router which directly does Modbus, GPS and VPN to my Home with HA. Unfortunately though it consumes about 15 Watts and during the winter months I have just enough solar for my VenusRPI :frowning:
Currently I am researching the best method to get VenusPi and RaspAP (with Adblockin, WireGuard to Home and Speedify to combine uplinks) running on one power efficient host (like a Pi4 running 3 raspberry systems as virtual machines). That would allow me a 24/7 uplink, some nice HA automations or even controlling some ESP32 for tank sensors and similar stuff.

1 Like

Hi Luis

Many thanks for your informative article on the Cerbo GX and MQTT broker. However, I already use MQTT broker in HA for Sonoff Light switches, etc, so I am not sure how to use HA with Cerbo GX in this scenario.

Can you please assist?

Many thanks in advance
Werner