Support for reading Dutch Smart Meter (electricity/gas) (P1 port)

I plan to make all meters to be supported by this component :).

I don’t have a screenshot at hand, maybe I can post one this evening.

Do you need the xonxoff setting to be 1? Or does 0 also work?

I made some changes to the underlying library of this component (dsmr_parser) which allows for better debugging and overriding of parity/bytesize settings. You can find it here: https://github.com/aequitas/dsmr_parser/tree/logging

To install it run:

pip3 install --upgrade https://github.com/aequitas/dsmr_parser/archive/logging.zip

This will install it separate from home-assistant so you can test the connection. To use it run:

dsmr_console

For debug output: dsmr_console -v, for help dsmr_console -h. To override parity for example run: dsmr_console -v -p E

If your meter readings work correct with this please post your settings here. I will convert them into settings for home-assistant. If multiple settings work for your device please post all settings that work (eg: EVEN and NONE parity). That way we can find the settings that work for everyone.

Had a look on this, but cant find out how to put the nauseate in there. Any guidance?

optional arguments:
-h, --help show this help message and exit
–device DEVICE port to read DSMR data from
–version {2.2,4} DSMR version (2.2, 4)
-v, --verbose increase output verbosity
-p {N,E,O}, --parity {N,E,O}
Override serial parity setting.
-b {7,8}, --bytesize {7,8}
Override serial bytesize setting.

0 works too I guess.

When I find the time I will test dsmr_parser.

When I contributed to smeterd, I passed some of my meter readings in for unit testing purposes, maybe you want to use then too:

See:
https://github.com/nrocco/smeterd/blob/master/tests/init.py

what do you mean by that word :smiley:

Currently my contribution to ndokters implementation doesn’t have that much testcases. I think I can use yours very well for that thx, or you could try adding them yourself? https://github.com/ndokter/dsmr_parser/blob/master/test/test_parse.py

Nauseate? You don’t know that? It is a typo :wink: and should be baudrate instead.

See for reference a sample telegram from my smart meter. Will check on all the other suggestions later this week.

/XMX5LGBBFG1012530239

1-3:0.2.8(42)
0-0:1.0.0(161116201035W)
0-0:96.1.1(4530303331303033323737383938383136)
1-0:1.8.1(000075.067kWh)
1-0:1.8.2(000084.813
kWh)
1-0:2.8.1(000000.000kWh)
1-0:2.8.2(000000.000
kWh)
0-0:96.14.0(0002)
1-0:1.7.0(02.778kW)
1-0:2.7.0(00.000
kW)
0-0:96.7.21(00002)
0-0:96.7.9(00000)
1-0:99.97.0(0)(0-0:96.7.19)
1-0:32.32.0(00000)
1-0:32.36.0(00000)
0-0:96.13.1()
0-0:96.13.0()
1-0:31.7.0(012A)
1-0:21.7.0(02.778
kW)
1-0:22.7.0(00.000kW)
0-1:24.1.0(003)
0-1:96.1.0(4730303435303031363038313037323136)
0-1:24.2.1(161116200000W)(00066.405
m3)
!CC1E

ah yes, I don’t think it differs between devices using the same protocol as the baudrate is defined in the DSMR itself.

2.2 is 9600
4.x is 115200

if you want to test dsmr_console with 115200 just run:

dsmr_console --version 4 

it defaults to protocol 2.2.

thanks for the sample, I added it to the test suite: https://github.com/aequitas/dsmr_parser/commit/9723e539ed2f5e6c9242b8b408f93b77dffccea8

Seems to work fine from a parsing point of view. So i think if you specify the version it should work fine. If not let me know.

Ok, I decided to remove all modules from Hass except the DSMR module. Apparently the data stops flowing in after a short while which leads to the overview below

In addition I use Domoticz to read the P1 port on the same system. In there i do get the expected results

Is there now something I am doing wrong? Could it, for example, be that reading the Same port in parallel through both hass and Domoticz causes conflicts?

I tested with reading my P1 by a Python script I found somewhere online, just to see if I could get it into InfluxDB. The moment I got values by script, Domoticz stopped communicating. So yeah, I think it’s not possible to do it in parallel. :neutral_face:

Functional for me:

Meter: Iskra ME382
DSMR expected: 2.2
P1 cable: ordered at https://www.slimmemeterkabel.nl/
Serial settings:

SERIAL_SETTINGS_V2_2 = {
    'baudrate': 9600,
    'bytesize': serial.SEVENBITS,
    'parity': serial.PARITY_EVEN,
    'stopbits': serial.STOPBITS_ONE,
    'xonxoff': 0,
    'rtscts': 0,
    'timeout': 20
}

Reading the serial device with two programs might cause some issues. But my implementation does not yet handle every kind of error so it might not recover like domoticz.

Can you see if there are any interesting messages in home-assistant.log file in you configuration directory?

Had a look in one of my old logs and totally nothing mentioned about the DSMR module. I did now remove Domoticz and data is flowing in relative nicely. It still looks like as if the gas readings are not very frequent (sometimes once every 40 mins) but I can live with that :smile:

Good to know it works now. Gas meter readings are indeed different. The smartmeter only reads them every 30/60 minutes or so from the gasmeter :frowning:

Yesterday evening I noticed that the readings stopped. The log reads the following:

16-11-18 20:03:54 homeassistant.core: Bus:Handling <Event state_changed[L]: old_state=<state climate.woonkamer=unknown; unit_of_measurement=°C, friendly_name=Woonkamer, min_temp=7, current_temperature=21.4, max_temp=35, temperature=21.5, away_mode=off @ 2016-11-18T16:14:29.477862+01:00>, new_state=<state climate.woonkamer=unknown; unit_of_measurement=°C, friendly_name=Woonkamer, min_temp=7, current_temperature=21.3, max_temp=35, temperature=21.5, away_mode=off @ 2016-11-18T16:14:29.477862+01:00>, entity_id=climate.woonkamer>
16-11-18 20:03:54 homeassistant.core: Bus:Handling <Event state_changed[L]: old_state=<state sensor.ram_use=43.3; icon=mdi:memory, unit_of_measurement=%, friendly_name=RAM Use @ 2016-11-18T21:00:31.482190+01:00>, new_state=<state sensor.ram_use=52.3; icon=mdi:memory, unit_of_measurement=%, friendly_name=RAM Use @ 2016-11-18T21:03:54.368816+01:00>, entity_id=sensor.ram_use>
16-11-18 20:03:54 homeassistant.core: Bus:Handling <Event state_changed[L]: old_state=<state sensor.power_consumption=3.005; icon=mdi:flash, unit_of_measurement=kW, friendly_name=Power Consumption @ 2016-11-18T21:03:54.033763+01:00>, new_state=<state sensor.power_consumption=3.015; icon=mdi:flash, unit_of_measurement=kW, friendly_name=Power Consumption @ 2016-11-18T21:03:54.444989+01:00>, entity_id=sensor.power_consumption>
16-11-18 20:03:54 homeassistant.core: Bus:Handling <Event state_changed[L]: old_state=<state sensor.power_consumption_low=83.923; icon=mdi:flash, unit_of_measurement=kWh, friendly_name=Power Consumption (low) @ 2016-11-18T21:03:54.055311+01:00>, new_state=<state sensor.power_consumption_low=83.933; icon=mdi:flash, unit_of_measurement=kWh, friendly_name=Power Consumption (low) @ 2016-11-18T21:03:54.511601+01:00>, entity_id=sensor.power_consumption_low>
16-11-18 20:03:54 homeassistant.core: Bus:Handling <Event state_changed[L]: old_state=<state sensor.power_consumption=3.015; icon=mdi:flash, unit_of_measurement=kW, friendly_name=Power Consumption @ 2016-11-18T21:03:54.444989+01:00>, new_state=<state sensor.power_consumption=2.999; icon=mdi:flash, unit_of_measurement=kW, friendly_name=Power Consumption @ 2016-11-18T21:03:54.873944+01:00>, entity_id=sensor.power_consumption>
16-11-18 20:03:54 homeassistant.core: Bus:Handling <Event state_changed[L]: old_state=<state sensor.power_consumption_low=83.933; icon=mdi:flash, unit_of_measurement=kWh, friendly_name=Power Consumption (low) @ 2016-11-18T21:03:54.511601+01:00>, new_state=<state sensor.power_consumption_low=83.940; icon=mdi:flash, unit_of_measurement=kWh, friendly_name=Power Consumption (low) @ 2016-11-18T21:03:54.904649+01:00>, entity_id=sensor.power_consumption_low>
16-11-18 20:03:54 homeassistant.core: Bus:Handling <Event state_changed[L]: old_state=<state sensor.power_consumption=2.999; icon=mdi:flash, unit_of_measurement=kW, friendly_name=Power Consumption @ 2016-11-18T21:03:54.873944+01:00>, new_state=<state sensor.power_consumption=2.998; icon=mdi:flash, unit_of_measurement=kW, friendly_name=Power Consumption @ 2016-11-18T21:03:54.937807+01:00>, entity_id=sensor.power_consumption>
16-11-18 20:03:54 homeassistant.core: Bus:Handling <Event state_changed[L]: old_state=<state sensor.power_consumption_low=83.940; icon=mdi:flash, unit_of_measurement=kWh, friendly_name=Power Consumption (low) @ 2016-11-18T21:03:54.904649+01:00>, new_state=<state sensor.power_consumption_low=83.948; icon=mdi:flash, unit_of_measurement=kWh, friendly_name=Power Consumption (low) @ 2016-11-18T21:03:54.970360+01:00>, entity_id=sensor.power_consumption_low>
16-11-18 20:03:55 custom_components.sensor.dsmr: error during initialization of DSMR serial reader: ("Invalid '%s' line for '%s'", '1-0:1.7.0(02.992\x00*kW)', <dsmr_parser.parsers.CosemParser object at 0xad7df870>)

In the mean time my own solution is aqcuiring data just nicely:


The total numbers are not so useful for these graphs, and the current power usage is not so useful for long-term analysis.

@aequitas did you take a look at the data in the database, and/or other graphing options in HASS?

I would like the have the option to look at power/gas consumption over a larger period in different resolutions. (like weekly usage over the last two years)
Other then other measures I’ve seen in the database, my meters return simple values which could be used for aggregations like this.

I know HASS uses SQL alchemy, but otherwise overview could be done like below:
(the @prev trick allows the previous record to be read cheaply in MySql and might not work in SqlLite)

select 
    DayOfWeek,
    Month,
    Day,
    Hour,
    Current-Previous
from (
	select 
		DayOfWeek,
		Month,
		Day,
		Hour,
		cast(@prev as Decimal(10,3)) as previous,
		@prev := e.maxstate as current
	from(
		select
			  (@prev:=null) as init
			, dayname(created) as DayOfWeek
			, month(created)   as Month
			, day(created)     as Day
			, hour(created)    as Hour
			, max(cast(state as decimal(10,3))) as maxstate
			from homeassistant_dev.states
		where entity_id in ('sensor.power_used__low_tariff', 'sensor.power_used__normal_tariff')
		group by month(created),      day(created),      hour(created)
		order by month(created),      day(created),      hour(created)
		) as e
	) as e2
order by Month desc, Day desc, Hour desc

This delivers me the hourly usage based on the total electricity low AND high meters:

Saturday	11	19	21	0.430
Saturday	11	19	20	0.504
Saturday	11	19	19	0.515
Saturday	11	19	18	0.452
Saturday	11	19	17	0.422
Saturday	11	19	16	0.601
Saturday	11	19	15	0.551
Saturday	11	19	14	0.267
Saturday	11	19	13	0.201
Saturday	11	19	12	0.323
Saturday	11	19	11	0.438
Saturday	11	19	10	0.281
Saturday	11	19	9	0.342
Saturday	11	19	8	0.871
Saturday	11	19	7	0.758
Saturday	11	19	6	0.229
Saturday	11	19	5	0.171
Saturday	11	19	4	0.190
Saturday	11	19	3	0.176
Saturday	11	19	2	0.184
Saturday	11	19	1	0.181
Saturday	11	19	0	0.175
Friday		11	18	23	-117.748
Friday		11	18	22	0.002
Friday		11	18	21	0.494
Friday		11	18	20	0.618
Friday		11	18	19	0.605
Friday		11	18	18	0.456
Friday		11	18	17	0.608
Friday		11	18	16	0.418
Friday		11	18	15	0.562
Friday		11	18	14	0.650
Friday		11	18	13	0.299
Friday		11	18	12	0.245
Friday		11	18	11	0.238
Friday		11	18	10	0.251
Friday		11	18	9	0.313
Friday		11	18	8	0.315
Friday		11	18	7	0.399
Friday		11	18	6	111.687
Friday		11	18	5	0.171
Friday		11	18	4	0.147
Friday		11	18	3	0.133
Friday		11	18	2	0.207
Friday		11	18	1	0.132
Friday		11	18	0	0.149
Thursday	11	17	23	-112.423
Thursday	11	17	22	0.002
Thursday	11	17	21	0.297
Thursday	11	17	20	0.496

This results in:


(I ignored the three outliers which I have not explained yet)

[edit:removed some spelling mistakes]

Got the same issue again. See log below:

16-11-20 09:48:21 custom_components.sensor.dsmr: error during initialization of DSMR serial reader: ("Invalid '%s' line for '%s'", '1-0:1.7.0(00.315\x00*kW)', <dsmr_parser.parsers.CosemParser object at 0xb63929f0>)

@Atreyu I use Emoncms on my webserver for that. It produces very nice graphs and also allows me to include a dashboard on my homepage. See some screenshots below. It integrates very well with Homeassitant (both uploading data as getting data through the emoncms.sensor and hisotry component)

My energy and temperature dashboard:

My temperature yearly graph:

I will have a look at that. It seems it doesn’t recover after a parsing failure. I have not encountered it myself before.

I use graphite/grafana for long term metric storage and analysis. https://home-assistant.io/components/graphite/ http://grafana.org/

1 Like

I read a few times about graphite/grafana, but it seemed/seems too hard to get it to work.

As you just mention it so casually, I gave it a try.

I now have Grafana up and running (empty) on my Pi 1B (don’t think it’s doable on my qnap NAS). The Grafana db server is now set to MySql (which is on my NAS, for more long-term storage).

Unfortunately it seems HASS can only pass data to Grafana using Graphite or InlfuxDb which for me are still obscure formats. (certainly not as mainstream as MySql).
For neither I have found clear installation instructions for a Pi1B.

Guess this is for another day.

On what hardware do you run your setup?