Support for reading Dutch Smart Meter (electricity/gas) (P1 port)

Nice! thanks for your work on this!

Great to see more people are working on making P1 port readings available in HA! I am currently working on getting the values to emoncms first and then sending them to HA through MQTT. As I am not a coder, you will most probably be much faster in getting a workable version in place. Let me know if I can test something.

1 Like

@Tyfoon @fversteegen, I pushed an updated feature complete version of the DSMR component this weekend and am now working on improving it for acceptance in HA. If you could test out the most recent version that would be great. Documentation abount the component can be found here: https://github.com/home-assistant/home-assistant.github.io/pull/1432/files

@aequitas I, unfortunately, need a little bit of guidance on how to install thisā€¦ Do is simply need to do git pull in some folder?

No problem, basically you just have to replace the package name with an url to my code version. So for the default instructions: https://home-assistant.io/getting-started/ replace ā€˜homeassistantā€™ with ā€˜https://github.com/aequitas/home-assistant/archive/dsmr.zipā€™. So

pip3 install --upgrade https://github.com/aequitas/home-assistant/archive/dsmr.zip

This will replace your currently installed version with the one with my changes.

If you just like to try it out without modifying your own installation try installing it in a virtualenv: https://home-assistant.io/getting-started/installation-virtualenv/ again replace ā€˜homeassistantā€™ with the url.

i justed dropped the dsmr.py in my ā€˜custom_components/sensor/ā€™ directory.
that worked for me :slight_smile:

the documentation states ā€˜deviceā€™ but should name ā€˜portā€™ instead by the way.

will see how this goes for a few daysā€¦ seems to work for now :wink:
thank you for taking time to make this

edit:
iā€™m not sure but it seems that the power_consumption_low and power_consumption_normal are reversed. Im on low tariff at the moment, but the normal sensor is updatedā€¦
i have a landis & gyr E350 DMSR 4.

Thanks for testing.

Yes, thats also possible. I just discovered HA last week so iā€™m not yet accustomed to all its features :slight_smile:

Should be ā€˜portā€™ indeed, thanks for reminding me to update the documentation.

Nor you mention it mine also seems reversed. The DSMR V2.2 document seems to have a conflict in explaining the tariff.

I will commit the changes to fix this in a few minutes.

Ok, I managed to test this. First the steps I took:

  1. Create folder /home/hass/.homeassistant/custom_components/sensor
  2. went to that folder and did sudo wget https://raw.githubusercontent.com/aequitas/home-assistant/dsmr/homeassistant/components/sensor/dsmr.py
  3. Included in my configuration.yaml
  • name: dsmr
    platform: dsmr
    port: /dev/ttyUSB-P1
    dsmr_version: 4
  1. Rebooted

When looking at the log I get:

16-11-14 11:56:29 homeassistant.core: Bus:Handling <Event state_changed[L]: new_state=<state sensor.power_consumption=0.276; unit_of_measurement=kW, icon=mdi:flash, friendly_name=Power Consumption @ 2016-11-14T12:56:29.909875+01:00>, old_state=<state sensor.power_consumption=unknown; icon=mdi:flash, friendly_name=Power Consumption @ 2016-11-14T12:56:14.701698+01:00>, entity_id=sensor.power_consumption>
16-11-14 11:56:30 homeassistant.core: Bus:Handling <Event state_changed[L]: new_state=<state sensor.power_production=0.000; unit_of_measurement=kW, icon=mdi:flash, friendly_name=Power Production @ 2016-11-14T12:56:30.038824+01:00>, old_state=<state sensor.power_production=unknown; icon=mdi:flash, friendly_name=Power Production @ 2016-11-14T12:56:14.861654+01:00>, entity_id=sensor.power_production>
16-11-14 11:56:30 homeassistant.core: Bus:Handling <Event state_changed[L]: new_state=<state sensor.power_tariff=normal; icon=mdi:flash, friendly_name=Power Tariff @ 2016-11-14T12:56:30.179945+01:00>, old_state=<state sensor.power_tariff=unknown; icon=mdi:flash, friendly_name=Power Tariff @ 2016-11-14T12:56:13.801596+01:00>, entity_id=sensor.power_tariff>
16-11-14 11:56:30 homeassistant.core: Bus:Handling <Event state_changed[L]: new_state=<state sensor.power_consumption_low=69.409; unit_of_measurement=kWh, icon=mdi:flash, friendly_name=Power Consumption (low) @ 2016-11-14T12:56:30.275386+01:00>, old_state=<state sensor.power_consumption_low=unknown; icon=mdi:flash, friendly_name=Power Consumption (low) @ 2016-11-14T12:56:13.932947+01:00>, entity_id=sensor.power_consumption_low>
16-11-14 11:56:30 homeassistant.core: Bus:Handling <Event state_changed[L]: new_state=<state sensor.power_consumption_normal=65.137; unit_of_measurement=kWh, icon=mdi:flash, friendly_name=Power Consumption (normal) @ 2016-11-14T12:56:30.316297+01:00>, old_state=<state sensor.power_consumption_normal=unknown; icon=mdi:flash, friendly_name=Power Consumption (normal) @ 2016-11-14T12:56:14.079811+01:00>, entity_id=sensor.power_consumption_normal>
16-11-14 11:56:30 homeassistant.core: Bus:Handling <Event state_changed[L]: new_state=<state sensor.power_production_low=0.000; unit_of_measurement=kWh, icon=mdi:flash, friendly_name=Power Production (low) @ 2016-11-14T12:56:30.436819+01:00>, old_state=<state sensor.power_production_low=unknown; icon=mdi:flash, friendly_name=Power Production (low) @ 2016-11-14T12:56:14.217854+01:00>, entity_id=sensor.power_production_low>
16-11-14 11:56:30 homeassistant.core: Bus:Handling <Event state_changed[L]: new_state=<state sensor.power_production_normal=0.000; unit_of_measurement=kWh, icon=mdi:flash, friendly_name=Power Production (normal) @ 2016-11-14T12:56:30.538800+01:00>, old_state=<state sensor.power_production_normal=unknown; icon=mdi:flash, friendly_name=Power Production (normal) @ 2016-11-14T12:56:14.337446+01:00>, entity_id=sensor.power_production_normal>
16-11-14 11:56:30 homeassistant.core: Bus:Handling <Event state_changed[L]: new_state=<state sensor.gas_consumption=57.707; unit_of_measurement=m3, icon=mdi:fire, friendly_name=Gas Consumption @ 2016-11-14T12:56:30.696445+01:00>, old_state=<state sensor.gas_consumption=unknown; icon=mdi:fire, friendly_name=Gas Consumption @ 2016-11-14T12:56:14.528492+01:00>, entity_id=sensor.gas_consumption>

So it appears to go right now :slight_smile:. I can confirm my meter is DSMR 4 and the port is correctā€¦ I do not get any gas metering in.

EDIT: whily typing I spotted my error. Changed the steps above and all appears to be working now :slight_smile:

Great to see this! Anyone know if this also works for DSMR 4.2? Or will I be the one to test that? :slight_smile:

Nice. Keep in mind that gas reading is only updating every hour (at least on my dsmr v2.2)

Could you tell me the brand/type of your smartmeter. I will add it to the list of supported devices.

Probably, but there could always be a minor difference in the protocol version. If you could be the guinea pig that would be very helpful :slight_smile:

As Koen noted, it is easier to just put the dsmr.py script in your custom_components folder. You can find it here:

https://raw.githubusercontent.com/aequitas/home-assistant/dsmr/homeassistant/components/sensor/dsmr.py

I donā€™t know if it works if you put it in <config dir>/custom_components/ or you need to put it in <config dir>/custom_components/sensor/

Very nice! Thank you so much for all the effort already.

Unfortunately, the parsing is not fully compatible with my meter yet. (Iskra ME 382, v2.2)

I receive the following errors:

Nov 14 16:39:11 raspberrypi hass[24824]: ERROR:homeassistant.components.sensor.dsmr:error during initialization of DSMR serial reader: (ā€œInvalid ā€˜%sā€™ line for ā€˜%sā€™ā€, ā€˜1-0:2.7.0(000Ar\x02\x02R[o]\x7fc\x0f0-ARā€™, <dsmr_parser.parsers.CosemParser object at 0x6ead7550>)

Nov 14 16:49:02 raspberrypi hass[1066]: ERROR:homeassistant.components.sensor.dsmr:error during initialization of DSMR serial reader: (ā€œInvalid ā€˜%sā€™ line for ā€˜%sā€™ā€, ā€˜0-0:96.1.1(4B413650303A\x1a*\x1aā€™, <dsmr_parser.parsers.CosemParser object at 0x6ce06810>)

Edit (solved!):

I solved this issue by changing the parity setting in the serial.py file of the DSMR parser. Works now with my setup! Thanks. See also: https://github.com/ndokter/dsmr_parser/issues/4

My landis & gyr e350 is sometimes referred to as 4.0/4.2. For now this sensors works on my meter.

Wow this thread has come to life! I was just picking up where I left off, made some progress and wanter to report it here, when I saw that @aequitas started too! Somehow I missed the notification.

@aequitas you seem far more tuned into Python that I am, and seem to be able to dedicate more time too. Are you about to wrap this up and make a pull request? Do you have any screenshot of what your setup now looks like?
You are also on a more recent version of home assistant and see to be using async methods.

I finished reading the data for both gas and electricity (I borrowed your icon-choice):

(the history needs to get accustomed to my bugfixes)

But there is still a some or work to do:

  • Implement async methods, limit reporting rate per sensor type.
  • Check the data format for each sensor.
  • Create (interactive) graphs for periodical reports.

Weird I have the same meter. I think it has something to do with the usr->serial device used.

When I switch my serial to EVEN parity it works as well as NONE parity. I have to meters of the ME 382 type to test and both work with EVEN parity.

There is nothing about the parity in the DSMR spec. But I think we can make EVEN the default setting.

https://github.com/ndokter/dsmr_parser/pull/5

@Atreyu Iā€™m a Python Software Engineer by trade so go figure :wink:

I have a pull request open on the HA repo with all todos finished. Iā€™m waiting for them to accept it: https://github.com/home-assistant/home-assistant/pull/4309

I donā€™t know when they introduces the asyncio stuff, I am a little new to HA just picked it up last week, but it seemed that it is the direction they wanted to go with new plugins and I wanted to learn it anyway :slight_smile:

If I may give you some advise on your module, you could go for asyncio if you want, but you cannot use smeterd in that case because it has no support for asyncio. I would advise to used a threaded approach using the old API (non ā€˜async_ā€™ functions) instead. Because otherwise homeassistant will be unresponsive during the time it needs to wait for a new packet to arrive. Another way to have non blocking serial IO is to use the pyserial function ā€˜in_waitingā€™ to check if there is data to be received and parsed. (Or just pull every 11 seconds or so so the serial buffer stays full, but that is a little dirty solution :))

I have got a Landis+Gyr serie ZCF110 / ZM F110 which apparently is DSMR 4.2. the script works for me after a reboot, but stops working after a while. Am trying to figure out in the log what is going wrong. Will let you know asap.

Edit: got a hunch it might be due the parity as indicated by @aequitas. See also this detailed, but Dutch, description here: http://domoticx.com/p1-poort-slimme-meter-hardware/

Its hard to figure out what the serial settings should be as they are not defined in the DSMR spec afaik. For mine it worked with both parities others might have some issues. I think it often depends on the usb<>serial adapter used.

Do you know how to edit the parity in source en test if you get a stable signal using N parity for your meter? IF not let me know.

Depending on the outcome I will try to find a solution for it. I will also try to add some debug logging information to the component which may help tracing these kind of issues.

Could everyone who has this component working with his/her setup please post:

  • smart meter brand/type name
  • DSMR version you expect this meter to be
  • usb<->serial device brand/type (if possible) and/or url to sellers site
  • serial settings which seem to work

I can then make this information available on the component help page.

@aequitas you are way ahead of me, and I hope your component will support my meter too.

Smeterd isnā€™t too stable, but that may be inherent to reading serial connections or my particular config.

Any screenshots of your setup? You may want to use some of my code for config validation.

I have a Kaifa meter, of which I think the model is: Kaifa E0026

using these settings, I mostly got successes:

baudrate = 115200
bytesize=serial.EIGHTBITS
parity=serial.PARITY_NONE
stopbits=serial.STOPBITS_ONE
xonxoff=1
rtscts=0
timeout=20
port="/dev/ttyUSB0"

Iā€™ve got a P1 Converter Cable v2, as bought from https://sites.google.com/site/nta8130p1smartmeter/webshop
I attached it to Raspberry Pi 1 B, suggested connection settings (as used above)
https://sites.google.com/site/nta8130p1smartmeter/blog/p1convertercablenutebestellenindewebshop/P1CC%20Info.txt?attredirects=0