Energy panel with heating in GJ

pooh22 · January 7, 2022, 4:37pm

I just received my “warmtelink” box on my heat-exchanger (“stadsverwarming”), which has a fully functional P1 port ^^

The units coming out of this port are in GJ, which I somehow need to get into the home assistant.

I’m very new to home assistant, started only 2 weeks ago to get an interface to my new shelly plugs, so please bear with me

At the moment I only have a working P1 script, but I would like to build this properly and try to make it so others can use it as well.

I’m planning to have a good look at what is available for the usual P1 application (electricity) and see if I can transform it to read GJ from my heating system. I’m using a raspberry-pi with now 2 p1 reading cables (it’s a bit tight), so it is already running production for my other energy. This is a custom script, which I extended with mqtt to get the readings into HA. The original way I used the P1 info is using icinga with pnp4nagios, and this still works and this actually triggers the p1 reading for electricity (and thus the mqtt publication).

For this new P1 thing, I think I might want something a little more “usual”, in the sense that I think most implementations use a daemon like system, which sends info every time a P1 packet arrives on the serial interface or after a certain interval (once a minute is plenty, perhaps even 5 minutes or longer for heating).

I want to log locally to a file, to be able to do something with the data in another way when e.g. my HA install goes south.

I believe currently HA doesn’t have the ability to see GJ as an energy unit, so I may need to do some calculations either in home assistant (template) or in the backend sending the data to HA. So far I didn’t have much luck with the template stuff, but that’s probably me doing it wrong, it would be my preference to use the templating, since HA might at some point learn about GJ and could use it directly…

I’m open to advice, suggestions or collaboration if anyone is interested…

Cheers

/Simon

pooh22 · January 8, 2022, 1:47pm

To follow up on this, I have a basic version working more or less.

on the rpi I’m using a python script to read the serial port and process the value, send it via mqtt:

dsmrdata = {}

for l in telegramlines:
    #os.system('clear')
    ##### 0-1:24.2.1(220108121022W)(0.578*GJ)
    #### YYMMDDhhmmssX ASCII presentation of Time stamp with Year, Month, Day,
    #### Hour, Minute, Second, and an indication whether DST is active (X=S) or DST
    #### is not active (X=W).
    meter_regex='0-1:24.2.1\((\d\d)(\d\d)(\d\d)(\d\d)(\d\d)(\d\d)([WS])\)\((\d+\.\d+)\*GJ\)$'
    meter_match=re.search(meter_regex, l)
    if meter_match:
        timestamp = "20{0}-{1}-{2} {3}:{4}:{5}".format(meter_match.group(1),meter_match.group(2),meter_match.group(3),meter_match.group(4),meter_match.group(5),meter_match.group(6))
        if meter_match.group(7) == "S":
            zomertijd="true"

        heat_gj = float(meter_match.group(8))
        heat_m3gas = heat_gj * GJgasfactor
        heat_kwh = heat_gj * GJtokWhfactor
        print "meterstand {0:6.3f}GJ equiv {1:7.3f}m3 gas om {2}".format(heat_gj,heat_m3gas,timestamp)
        dsmrdata["heat_gj"] = heat_gj
        dsmrdata["heat_m3gas_calculated"] = heat_m3gas
        dsmrdata["heat_kwh_calculated"] = heat_kwh
        dsmrdata["timestamp"] = timestamp
        data_json=json.dumps(dsmrdata)
        client.connect(mqttbroker) # connect must be just before publish 
        client.publish("energy/dsmr/heat_meter", data_json)
        client.disconnect() #

I was having some issues getting the measurements into HA, but after making the state_class total_increasing it showed up in the gas meter part of the energy panel.

- platform: mqtt
  state_topic: "energy/dsmr/heat_meter"
  name: "dsmr_kwh_calculated"
  value_template: "{{ value_json.heat_kwh_calculated }}"
  unit_of_measurement: 'kWh'
  device_class: energy
  state_class: measurement
- platform: mqtt
  state_topic: "energy/dsmr/heat_meter"
  name: "dsmr_gasmeter_calculated"
  value_template: "{{ value_json.heat_m3gas_calculated }}"
  unit_of_measurement: "m³"
  device_class: gas
  state_class: total_increasing
- platform: mqtt
  state_topic: "energy/dsmr/heat_meter"
  name: "dsmr_heatmeter"
  value_template: "{{ value_json.heat_gj }}"
  unit_of_measurement: 'GJ'
  state_class: measurement
- platform: mqtt
  state_topic: "energy/dsmr/heat_meter"
  name: "dsmr_heatmeter_timestamp"
  value_template: "{{ value_json.timestamp }}"
  device_class: timestamp

This is not a complete description, but in general this is what I did.

pooh22 · January 11, 2022, 7:31am

Apparently, my “warmtelink” has stopped sending information over the P1 port, so until I figure out if/when/how I can have access again, this experiment is on hold…

pooh22 · January 21, 2022, 2:38pm

I heard Today that the “warmtelink” boxes in general are turned off by Vattenfall until all the customers have been able to give consent or reject the use of their information. Until that is sorted out, the box is off, and both me and them won’t get information out of it.

Meanwhile, it would be really great if the energy panel would learn to accept GJ as a valid measure for heating energy, in addition to m³ gas (which is going to be obsolete in say 30 years anyway.

itsSaad · July 7, 2022, 12:20pm

I’m in the same boat as you but six months later .
I’m using ser2net to connect the P1 device to home assistant with the dsmr integration. Mostly it works. But I get rejected telegrams in the HA logs. Quite annoying.
On top of the integration I use a template to convert the GJ to m3.
This setup is useable to some extent. But these are the problems.

after every restart of HA the value of the templated sensor gets unknown. This creates wrong graphs in the energy dashboard. And the total costs show as a couple thousand euros
CRC Errors on the telegram

Will look at the python script above and try to use that instead of the dsmr integration.

pooh22 · November 13, 2022, 7:32pm

An update on the whole situation…

My letter came, so since November 1st my warmtelink is read by vattenfall, but also it should spit out meter values at least every 15 minutes.

It took me a while to get back to the point where I was when the meter was just installed, as I had apparently switched P1 cables. After a lot of trial and error I got my setup to where it was again.

The main thing I learned was that when using screen to read the serial port, you should specify baudrate; screen /dev/ttyUSB0 115200 (and quit with ctrl-a \

The latest release has given me another new issue, because now GJ is a supported unit for heat, howerver my HA doesn’t let me choose the energy value with that unit…:

- state_topic: "energy/dsmr/heat_meter"
  name: "dsmr_kwh_calculated"
  value_template: "{{ value_json.heat_kwh_calculated }}"
  unit_of_measurement: 'kWh'
  device_class: energy
  state_class: measurement

- state_topic: "energy/dsmr/heat_meter"
  name: "dsmr_gasmeter_calculated"
  value_template: "{{ value_json.heat_m3gas_calculated }}"
  unit_of_measurement: "m³"
  device_class: gas
  state_class: total_increasing

- state_topic: "energy/dsmr/heat_meter"
  name: "dsmr_heatmeter_gj"
  value_template: "{{ value_json.heat_gj }}"
  unit_of_measurement: 'GJ'
  device_class: energy
  state_class: measurement

- state_topic: "energy/dsmr/heat_meter"
  name: "dsmr_heatmeter_timestamp"
  value_template: "{{ value_json.timestamp }}"
  device_class: timestamp

Somehow I can only choose the calculated m³ gas equivalent as a heat input in the energy dashboard. Did I do something wrong or is the GJ feature not quite solidly integrated?

pooh22 · January 1, 2023, 12:54pm

I created a feature request topic to be able to add/use an alternative heat energy graph card to the energy dashboard. Please vote for it if you have an alternative to gas/electric heating installation!