Support for reading Dutch Smart Meter (electricity/gas) (P1 port)

Hi @aequitas, first of all thanks for your effort for getting the dutch smart meter supported in Home Assistant.

I have installed HA in a virtual environment on a RPi using Ubuntu Jessie (and the RPi All-in-One installer), but have some issues getting your software running. It seems that HA is not able to access the GPIO pins to which the cable is connected:

16-11-27 21:49:15 homeassistant.components.sensor: Error while setting up platform dsmr
Traceback (most recent call last):
  File "/home/hass/.homeassistant/deps/serial/serialposix.py", line 265, in open
    self.fd = os.open(self.portstr, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
PermissionError: [Errno 13] Permission denied: '/dev/ttyAMA0'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/hass/hass_venv/lib/python3.4/site-packages/homeassistant/helpers/entity_component.py", line 145, in _async_setup_platform
    entity_platform.async_add_entities, discovery_info
  File "/srv/hass/hass_venv/lib/python3.4/site-packages/homeassistant/components/sensor/dsmr.py", line 109, in async_setup_platform
    transport, _ = yield from hass.loop.create_task(dsmr)
  File "/usr/lib/python3.4/asyncio/futures.py", line 388, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.4/asyncio/tasks.py", line 286, in _wakeup
    value = future.result()
  File "/usr/lib/python3.4/asyncio/futures.py", line 277, in result
    raise self._exception
  File "/usr/lib/python3.4/asyncio/tasks.py", line 237, in _step
    result = next(coro)
  File "/usr/lib/python3.4/asyncio/coroutines.py", line 141, in coro
    res = func(*args, **kw)
  File "/home/hass/.homeassistant/deps/serial_asyncio/__init__.py", line 409, in create_serial_connection
    ser = serial.serial_for_url(*args, **kwargs)
  File "/home/hass/.homeassistant/deps/serial/__init__.py", line 88, in serial_for_url
    instance.open()
  File "/home/hass/.homeassistant/deps/serial/serialposix.py", line 268, in open
    raise SerialException(msg.errno, "could not open port {}: {}".format(self._port, msg))
serial.serialutil.SerialException: [Errno 13] could not open port /dev/ttyAMA0: [Errno 13] Permission denied: '/dev/ttyAMA0'

My vritual port is owned by the group tty:

hass@raspberrypi:/home/pi$ ls -la /dev/ttyAMA0
crw--w---- 1 root tty 204, 64 Nov 27 18:58 /dev/ttyAMA0

My virtualenv user is added to the tty group (as well as the gpio and dialout):

hass@raspberrypi:/home/pi$ groups | grep tty
hass tty dialout gpio

If I try as a different user (also memeber of tty) it seems it does work, but only when I elevate using sudo I get dsmr_console to properly access the port and report the data from the datagram(s):

pi@raspberrypi:~ $ ls -la /dev/ttyAMA0
crw--w---- 1 root tty 204, 64 Nov 27 18:58 /dev/ttyAMA0
pi@raspberrypi:~ $ groups | grep tty
pi adm tty dialout cdrom sudo audio video plugdev games users input netdev gpio i2c spi
pi@raspberrypi:~ $ dsmr_console --help
usage: dsmr_console [-h] [--device DEVICE] [--version {2.2,4}] [-v]
                    [-p {N,E,O}] [-b {7,8}]

Output DSMR data to console.

optional arguments:
  -h, --help            show this help message and exit
  --device DEVICE       port to read DSMR data from
  --version {2.2,4}     DSMR version (2.2, 4)
  -v, --verbose         increase output verbosity
  -p {N,E,O}, --parity {N,E,O}
                        Override serial parity setting.
  -b {7,8}, --bytesize {7,8}
                        Override serial bytesize setting.
pi@raspberrypi:~ $ dsmr_console --device /dev/ttyAMA0 --version 2.2
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/serial/serialposix.py", line 265, in open
    self.fd = os.open(self.portstr, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
PermissionError: [Errno 13] Permission denied: '/dev/ttyAMA0'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/dsmr_console", line 9, in <module>
    load_entry_point('dsmr-parser==0.3', 'console_scripts', 'dsmr_console')()
  File "/usr/local/lib/python3.4/dist-packages/dsmr_parser/__main__.py", line 63, in console
    for telegram in serial_reader.read():
  File "/usr/local/lib/python3.4/dist-packages/dsmr_parser/serial.py", line 63, in read
    with serial.Serial(**self.serial_settings) as serial_handle:
  File "/usr/local/lib/python3.4/dist-packages/serial/serialutil.py", line 236, in __init__
    self.open()
  File "/usr/local/lib/python3.4/dist-packages/serial/serialposix.py", line 268, in open
    raise SerialException(msg.errno, "could not open port {}: {}".format(self._port, msg))
serial.serialutil.SerialException: [Errno 13] could not open port /dev/ttyAMA0: [Errno 13] Permission denied: '/dev/ttyAMA0'
pi@raspberrypi:~ $ sudo dsmr_console --device /dev/ttyAMA0 --version 2.2
Power Consumption (low) 5782.433 kWh
Power Consumption (normal) 4304.307 kWh
Power Consumption 0.18 kW
Power Tariff 0001 None
Power Production 0.00 kW
Power Production (low) 0.000 kWh
Power Production (normal) 0.000 kWh

Power Consumption (low) 5782.434 kWh
Power Consumption (normal) 4304.307 kWh
Power Consumption 0.20 kW
Power Tariff 0001 None
Power Production 0.00 kW
Power Production (low) 0.000 kWh
Power Production (normal) 0.000 kWh

^CTraceback (most recent call last):
  File "/usr/local/bin/dsmr_console", line 9, in <module>
    load_entry_point('dsmr-parser==0.3', 'console_scripts', 'dsmr_console')()
  File "/usr/local/lib/python3.4/dist-packages/dsmr_parser/__main__.py", line 63, in console
    for telegram in serial_reader.read():
  File "/usr/local/lib/python3.4/dist-packages/dsmr_parser/serial.py", line 67, in read
    line = serial_handle.readline()
  File "/usr/local/lib/python3.4/dist-packages/serial/serialposix.py", line 472, in read
    ready, _, _ = select.select([self.fd, self.pipe_abort_read_r], [], [], timeout.time_left())
KeyboardInterrupt
pi@raspberrypi:~ $

Do you have any clue what could by the issue and how to get HA to read the DSMR data from inside the virtualenv?

Could you try a sudo chmod 666 /dev/ttyAMA0 and see if that improves the situation? I had similar issues with my raspberry in the beginning and this seemed to solve things.

Thanks for the hint @fversteegen, but security wise that is a bad suggestion as that would mean everybody is allowed to read and write to the device.
Apart from that I already tried changing permissions but they do not stick and are reset (periodically, most likely because of udev rules).

I figured it out this night. Due to the fact that the serial console was not properly disabled at startup it kept staring, despite disabling it using ‘sudo raspy-config’.
It seems permission changes to rw--w---- and ownership to root:try by getty, which provides the console login screen.

After disabling it by masking it like this:

sudo systemctl mask [email protected]
sudo reboot

Only after above commands the service does not start anymore and because of that permissions are rw-rw---- and ownership root:dialout.
Since my user running Home Assistant was already a member of the dialout group everything worked as expected after this.

It seems smart meter data is recorded nicely during the night.

1 Like

If nobody is having any issues with it then I am considering the gas usage ready. Hope it can be included in the next release with DSMR as well.

2 Likes

All running fine and apparently stable here @aequitas! Once more: many thanks for this great piece of work!

@aequitas Hi Johan, did you have a chance to look at my Q (Support for reading Dutch Smart Meter (electricity/gas) (P1 port)) . Quite handy as it allows you to move the HASS server to a more central location in the house.

I have a P1 Wifi cable (http://romix.macuser.nl/) which sends an identical report but then via the network. Instead of a USB port the plugin would need to check a IP/port combination.

Sorry totally missed your post there.

I think its totally doable. I already have it implemented for a different component I am working for.

I still didn’t manage to get your script up and running. The problem is in getting the sensor value into the script. If I trim down your script to:

import homeassistant.remote as remote
api = remote.API('127.1.0.1', '')
print(remote.get_state(api, 'sensor.gas_consumption.state'))

I get a value “None” in return. Any clue on how to fix that?

What steps should I need to do so I can test this component.

Would like to add it ASAP.

Just add this file to custom_components/sensor/ https://github.com/aequitas/home-assistant/blob/dsmr/homeassistant/components/sensor/dsmr.py

Don’t forget to remove it when the new version of HA is released, as that includes the sensor natively.

And what do I need to put in the configuration file?

documentation can be found here: https://github.com/home-assistant/home-assistant.github.io/blob/next/source/_components/sensor.dsmr.markdown

1 Like

Do I understand correctly from the documentation that the current usage is not reported?

It is reported…
sensor.power_consumption :slight_smile:

Great work aequitas! Thanks!

Now I need to figure out a way to get daily usage out of Influx :blush:

The latest version 0.34 now contains DSMR: https://home-assistant.io/components/sensor.dsmr/

The hourly gas usage and tcp/ip support will probably be available next release.

1 Like

Goodmorning all!

just intalled version .34 but i’m unable tot get the data in HA. manually search:

pi@raspberrypi:~ $ cu -l /dev/ttyUSB0 -s 115200 --parity=none
Connected.
/XMX5LGBBFFB231249762

1-3:0.2.8(42)
0-0:1.0.0(161204111652W)
0-0:96.1.1(4530303034303031363730313631383135)
1-0:1.8.1(001955.237*kWh)
1-0:2.8.1(000000.000*kWh)
1-0:1.8.2(002184.711*kWh)
1-0:2.8.2(000000.000*kWh)
0-0:96.14.0(0001)
1-0:1.7.0(00.438*kW)
1-0:2.7.0(00.000*kW)
0-0:96.7.21(00001)
0-0:96.7.9(00000)
1-0:99.97.0(0)(0-0:96.7.19)
1-0:32.32.0(00000)
1-0:32.36.0(00000)
0-0:96.13.1()
0-0:96.13.0()
1-0:31.7.0(002*A)
1-0:21.7.0(00.438*kW)
1-0:22.7.0(00.000*kW)
0-1:24.1.0(003)
0-1:96.1.0(4730303332353631323330323535363135)
0-1:24.2.1(161204110000W)(02661.821*m3)
!4DD7
~.

ìn the config file:

sensor slimme meter:
  - platform: dsmr

tried it also with

sensor slimme meter:
  - platform: dsmr   
    dsmr_version: 4

anyone an suggestion?

Can you check the home-assistant.log file for any error messages?

By default the 4 protocol is configured for Even parity. Can you test if you get data using that manually?

What output does this command give: dsmr_console --device /dev/ttyUSB0 --version 4

$ bash: dsmr_console: command not found

@fversteegen
what do you see in the frontend for the device under developer tools >> states?
is the state filled there?

I do see them,just made a group, but all info is unknown