Victron VRM Portal API data integration

lsgv · December 23, 2017, 8:34pm

! EDIT 15 May 2023!

This topic was opened 5 years ago and except in very particular circumstances is not useful anymore so I wanted to close it but can’t find a way to do that on the forum.
On the subject of integrating Victron equipment with HA here is a short summary for those ending up here:

Use the VRM API as explained in this thread
When? If your Victron installation is remote and it’s only connection to the “world” is via VRM
Use MQTT as explained in many threads and specifically here
When? If you’re familiar with MQTT and envision the use of, for instance, Red-Node, for logic programming and automations
Use modbusTCP either using DYI config or using the excellent HACS integration from @sfstar you can find here
When? If you’re familiar with modbus and like to keep it simple

After having used all 3 methods I am now and for the foreseeable future using modbus because I have other modbus integrations in my HA such as for instance swimming pool filters/clorinator, Fronius inverter, etc. which reduces the number of integration and keeps it simple. But this is a purely personal choice, feel free to test and try and break and share your experiences.

If you want Victron’s opinion please read this post by Matthys Vader on the subject. you can find the complete thread here

Original thread:

I have a Victron set-up in an off-grid solar pumping irrigation system with battery monitor BMV and VRM Victron Remote Management connection via a Color Control GX.
With this set-up, I have access to solar panels and battery data via the Victron VRM Portal. And using VRM API VRM API documentation? I have managed to access the data I want with POST and GET commands using Postman (I have little programming experience).

What I want to do now is to get this data (for instance, battery state of charge) into Home Assistant (I’m using hassio on a Raspberry PI) and use it to automate alarms (e.g. send message when state of charge drops below X) and actions (e.g. turn-off load when battery state of charge drops below X).

What do I need to do this? Any help or advice most welcome.

fabaff · January 6, 2018, 1:57pm

Take a look at the rest platforms. They done the same as you do with Postman.

pblgomez · May 20, 2018, 3:02pm

Did you get it to work?

lsgv · May 20, 2018, 8:56pm

Hi Pablo.
Yes, I did get it to work. took me some time to figure everything out but it’s running now for months with no issues.

Here is the short tutorial (first time I do this here so apologies if I don’t get formatting and everything correct, still earning…)

First, I used Postman to get acquainted with the API calls from VRM. You can fond these in https://vrmapi.victronenergy.com/v2/docs

The first one you need to use is the Login POST call https://vrmapi.victronenergy.com/v2/auth/login using username and password in the body. This gives you back a very long token (and your user ID, keep it for later) which you need to use when making other calls.

You have a screenshot of the Postman call here (user, passed, token and ID blanked out but you get the gist).

I found out this token has a life and after days (I don’t remember exactly how many, it appeared to expire). So the solution was to use the “personal access token” call

This returns a token which is valid forever (mine has been valid for months now). Attention! Make sure you copy the token when you get it because there is no way to retrieve it again. You can then use this token in all the other calls.

Here are the screenshots. They cover the same call but one is for the header where you need to put the long token, the second is for the body where you need to put in the name of the personal access token you will get (any name will do).

There are many calls available but what I wanted was a real-time read of the main parameters. The best one I found is https://vrmapi.victronenergy.com/v2/installations/your_installation_number_here/diagnostics?count=1000

The count defines the number of records you get, and 1000 is ok. The advantage is that with one call, you get everything you need.

Here is the screenshot of the call and the result. You use here the token retrieved earlier.

The next hurdle is to be able to use those data in home assistant using templates. Next post…

lsgv · May 20, 2018, 9:22pm

Tutorial on the Victron VRM connection - follow-up and finish.

Once I got the VRM API calls sorted out using Postman, I needed to be able to use the data returned using templating. That took quite a while!

Here is the configuration part of the yaml file (I put the calls to VRM into a dedicated sensor file).

You can see the value_template used.

Next you need to retrieve all the specific values you want. Which values are present in the return data depend on what equipment you have in your installation, but it is all very self-explanatory. You can check them using Postman.

Here is the configuration for a few sensor values.

Note the remarks I made to myself. I knew I would forget those details…
Some data values have units, other don’t. Those with units you need to transform them by taking out the units plus one space, and transforming into float. If you don’t do this, they’ll be strings and you cannot manipulate or use them as numbers. And you need to indicate the unit of measurement.

Second, the data values come back as an array, and to retrieve the nth element in the array you need to ask for the nth-1 because the first element in the array is numbered 0 (as in zero).

I have installed influx and feed these sensor values to it, and use kapacitor and chronograf for visualisation, alerts, etc. I also configured a couple of automations in hass to send me a message when battery charge reaches 100%, or when PV voltage is close to 125 V (the voltage limit of my MPPT is 150 V so I want a warning).

I can say this is working as well as I could want it and am now moving on for further automations and stuff.

It took me quite a lot of trial and error but persistence paid. And I learned a lot which accelerated doing the same with my irrigation controller GreenIQ.

The other great thing is the portability of all this. I started out tinkering on my mac, moved to a RPI with hass.io, wanted some more robust stuff and single hosting so moved on to a bespoke server rack with ESXi, Ubuntu VMs, and running hass, influx, kapacitor, cronograf, letsencrypt, etc. on containers. Also put an NGINX remote proxy in front of all this to be sure I don’t open up too many thing to the internet. And a VPN which I can use when away to SSH in case I need to repair or change anything. And I all just worked perfectly by copying the config files from one to the other. Magic!

Finally, here is a screenshot of the result (dedicated pages for main stuff like the solar tracking).

Hope this helps. Have fun, I sure did

frank451 · May 21, 2018, 7:15am

Hi

I have a victron setup, but i dont use the GX management portal. I have bluetooth comms (currently), if there was a way to get it into HA via serial/ethernet i would change over.

Do you know if a local connection is possible, and how i would read the data coming from my solar controller?

thanks

lsgv · May 21, 2018, 8:45am

Hi Frank.
The different Victron equipment units communicate using mostly open protocols, but using the CCGX has the advantage of plug-and-play and I wouldn’t be able to use the raw protocols myself.
I know for sure there is a way to use a Raspberry PI to create an equivalent of the CCGX and hence use it as platform onto and from which you can get all the data of your controller.

Just check out the link https://www.victronenergy.com/live/open_source:ccgx:installing_ccgx_func_on_raspberry_pi

Rgs.
Luis

pblgomez · May 21, 2018, 12:12pm

I do get now: 2018-05-21 11:54:53 ERROR (MainThread) [homeassistant.components.sensor.template] Could not render template Battery Current: UndefinedError: ‘mappingproxy object’ has no attribute ‘records’

and on my sensors i only get:
sensor.vrm_data 1526207307 friendly_name: vrm data

lsgv · May 21, 2018, 12:33pm

Could you get the Postman calls to work? And did you check the structure of the response data? I assume that would be similar to mine, but better check.

frank451 · May 21, 2018, 12:39pm

I’ve just found this which is a recent addition to the pi open source folks.

VictronPi
VictronPi Github

This uses direct usb cable connections to the Victron VE Bus devices. Might be helpful and possibly easier?, to pull the info from this, not sure how, im no python or nodejs expert

Cheers

pblgomez · May 21, 2018, 1:11pm

Yes, the Postman works perfect but the id’s are changed that’s why on my system voltage is 52 and current is 53.

lsgv · May 21, 2018, 1:50pm

Can you paste the first part of the response? That’s where you can see the structure of the JSON

pblgomez · May 21, 2018, 2:37pm

{
"success": true,
"records": [
    {
        "idSite": XXXXX,
        "timestamp": 1526207307,
        "Device": "Gateway",
        "instance": 0,
        "idDataAttribute": 1,
        "description": "gatewayID",
        "formatWithUnit": "%s",
        "dbusServiceType": null,
        "dbusPath": null,
        "code": "d",
        "formattedValue": "Venus OS",
        "dataAttributeEnumValues": [
            {
                "nameEnum": "VGR, VGR2 or VER",
                "valueEnum": 0
            },
            {
                "nameEnum": "Venus OS",
                "valueEnum": 1
            },
            {
                "nameEnum": "Venus OS",
                "valueEnum": 2
            },
            {
                "nameEnum": "LoRaWAN lopy",
                "valueEnum": 3
            },
            {
                "nameEnum": "PHYSEE NodeMCU",
                "valueEnum": 4
            }
        ],
        "id": 1
    },
    {
        "idSite": XXXXX,
        "timestamp": 1526207307,
        "Device": "Gateway",
        "instance": 0,
        "idDataAttribute": 2,
        "description": "Fw Version",
        "formatWithUnit": "%s",
        "dbusServiceType": null,
        "dbusPath": null,
        "code": "v",
        "formattedValue": "v2.15",
        "id": 2
    },

pblgomez · May 23, 2018, 10:50am

Do you need any more info?
Thanks

lsgv · May 23, 2018, 12:05pm

I’ll take a look at the response you got and check if it matches the one I have. Later today, at work now

lsgv · May 23, 2018, 6:14pm

Your data has the same structure as mine so I don’t understand why the template can’t render.
What do you see in the home assistant entity page for the vrm_sensor?

pblgomez · May 24, 2018, 2:05pm

Nothing ;(

Screenshot_20180524_150441727×278 9.96 KB

lsgv · May 27, 2018, 3:41pm

Intersting. It does get the timestamp correct which is in the first record [0], but doesn’t seem to get the other records. I’m afraid I don’t know what the issue could be.
Which editor are you using for the yaml file?

pblgomez · May 27, 2018, 4:37pm

Atom, I don’t think I misspelled it but it’s the first time I check an array

pblgomez · June 2, 2018, 12:36pm

Can I give you the login and you try it?