I will have a look at that. It seems it doesn’t recover after a parsing failure. I have not encountered it myself before.
I use graphite/grafana for long term metric storage and analysis. https://home-assistant.io/components/graphite/ http://grafana.org/
1 Like
I read a few times about graphite/grafana, but it seemed/seems too hard to get it to work.
As you just mention it so casually, I gave it a try.
I now have Grafana up and running (empty) on my Pi 1B (don’t think it’s doable on my qnap NAS). The Grafana db server is now set to MySql (which is on my NAS, for more long-term storage).
Unfortunately it seems HASS can only pass data to Grafana using Graphite or InlfuxDb which for me are still obscure formats. (certainly not as mainstream as MySql).
For neither I have found clear installation instructions for a Pi1B.
Guess this is for another day.
On what hardware do you run your setup?
Both graphite is the de-facto standard in the industry for storing time series data so I wouldn’t call it obscure ;). It is a different database type then mysql because mysql is not optimal for storing this kind of data.
Graphite consist of 2 parts: carbon (data collection and storage) and graphite (web interface to display metrics). For carbon I would suggest installing: https://github.com/lomik/go-carbon which is a little bit more efficient then the default carbon and for graphite you can choose: http://graphite.readthedocs.io/en/latest/install.html or https://graphite-api.readthedocs.io/en/latest/installation.html
I have run a complete graphite + grafana setup on a RPi 1B before, it should be doable, although I don’t know if you could fit graphite + hass on one RPi memory wise.
Can you try installing this version: pip3 install https://github.com/aequitas/dsmr_parser/archive/error_handling.zip --target .homeassistant/deps/ --upgrade
It does not solve the parse problem, but it should prevent reading to cease after a parse failure.
Also I think the DSMR 4 includes a CRC at the end of the telegram, after the !. I have not searched in the spec if that is actually the case. But if it is maybe we could use it to determine if a telegram is valid before parsing it.
I changed the internals of the component a little bit to provide better integration with asyncio. Newest version can be found here to be installed in your custom_components directory: https://raw.githubusercontent.com/aequitas/home-assistant/dsmr/homeassistant/components/sensor/dsmr.py
@fversteegen: this change includes the error handling I worked on earlier.
@aequitas Hi Johan, would this component also work via a wifi enables P1 (http://romix.macuser.nl/ )
Good progress made here I notice! Thanks for the hard work! In the pull request I noticed it’s probably going to make 0.34. That’s great! 
I was wondering, does it also give us current usage sensors? If not, it shouldn’t be to hard to code that in, right?
Again, thanks for the hard work guys 
Thanks @aequitas! Trying the fix right now.
@ultrasub also the current electricity usage but only the total gas reading. Am working on breaking this down into the hourly usage.
Edit: the new version doesn’t appear to work as expected. See the log below:
16-11-21 20:06:09 dsmr_parser.protocol: failed to parse telegram
Traceback (most recent call last):
File “/home/hass/.homeassistant/deps/dsmr_parser/protocol.py”, line 82, in handle_lines
parsed_telegram = self.telegram_parser.parse(self.telegram)
File “/home/hass/.homeassistant/deps/dsmr_parser/parsers.py”, line 32, in parse
obis_reference, dsmr_object = self.parse_line(line_value.strip())
File “/home/hass/.homeassistant/deps/dsmr_parser/parsers.py”, line 47, in parse_line
return obis_reference, parser.parse(line_value)
File “/home/hass/.homeassistant/deps/dsmr_parser/parsers.py”, line 135, in parse
return CosemObject(self._parse(line))
File “/home/hass/.homeassistant/deps/dsmr_parser/parsers.py”, line 84, in _parse
raise ParseError(“Invalid ‘%s’ line for ‘%s’”, line, self)
dsmr_parser.exceptions.ParseError: (“Invalid ‘%s’ line for ‘%s’”, ‘\x7f05\x02RJ2r)S&\x170!J)Ha15\x02R\x1a\nr:r\r\x13%\x1802A%jHx15\x02R\x12\nri\x13\t&J009\x1a"RikUJC\n15\x02R\x12\x12ri\x13\t&J009\x02\x02\x02RikUJC\n05\nR\x12"r)\x13\t&J00MJ)Ha05\nRJ2r)\x13\t&J47M\x02\x1a\x02\x1a"\x1a\x1a\x02\x1a\x02\x1a\n\x1a2\x1a\x02\x1aB\x1a\n\x1a\x02\x1a:\x1a\x12\x1a\n\x1a2Jjlx05\nR\x12"rI\x13)&J16E\n\x12\n\x12\x02\x02\x02\x02\x02:\x15\t%L008ar\t\x13L&\x15mMJ)Ha!\x19\x15\x05"jlx/a5\x05U\x13j\x14(!FG10E\x12\x1a\x02\x12\x1aJjH(Ha1-3:0.2.8!"\x12J)Hx05\x02R\nr\t\x13\t&J16E\n\x12\n\x12\n\x02\x12*\x12:\x15)Ha05\x02RJ2r\nr)\x13J\x1a53A\x1a\x02\x1a\x1a\x1a\n\x1a\x02\x1a\x02\x1a\x1a\x1a\x12\x1a:\x1a:\x1aB\x1aJ\x1aB\x1aB\x1a\n\x1a2J)Ha15\x02R\nr\tr)\x13J\x180A\n\n"r\tB2Rik\x15ZJ’, <dsmr_parser.parsers.CosemParser object at 0xad5beed0>)
16-11-21 20:14:30 dsmr_parser.protocol: failed to parse telegram
Traceback (most recent call last):
File “/home/hass/.homeassistant/deps/dsmr_parser/protocol.py”, line 82, in handle_lines
parsed_telegram = self.telegram_parser.parse(self.telegram)
File “/home/hass/.homeassistant/deps/dsmr_parser/parsers.py”, line 32, in parse
obis_reference, dsmr_object = self.parse_line(line_value.strip())
File “/home/hass/.homeassistant/deps/dsmr_parser/parsers.py”, line 47, in parse_line
return obis_reference, parser.parse(line_value)
File “/home/hass/.homeassistant/deps/dsmr_parser/parsers.py”, line 135, in parse
return CosemObject(self._parse(line))
File “/home/hass/.homeassistant/deps/dsmr_parser/parsers.py”, line 84, in _parse
raise ParseError(“Invalid ‘%s’ line for ‘%s’”, line, self)
dsmr_parser.exceptions.ParseError: (“Invalid ‘%s’ line for ‘%s’”, ‘\x7f0-ARJ2r)\x13M\x170!\x02\x02\x02\nJjlx15\x02R\nri\x13\t&J00.M"*RikUJC\x0f15\x02R\x12ri\x13\t&J009\x02\x02\x02RZkUJC\x0f0-ARJ2ri\x13I&L(00A\x02\x12J)Ha0-0:96.7.9!\x02\x02\x02\x02\x02J)Ha15\x02RJJrJ:r\t\x13J\x18)!\x02j\t\x13’N6.]r)SNJ’, <dsmr_parser.parsers.CosemParser object at 0xad5f0270>)
Hurray I’ve got some decent dashboarding going on using InfluxDb and Grafana!
Influxdb was easy to install, just a small hiccup where they didn;t very abondantly mention that the web ui has been deprecated.
Grafana took some more effort, but was done using https://github.com/fg2it/grafana-on-raspberry
Now this has been set up, I eagerly await your pull request to be accepted.
Like @fversteegen mentioned, for power usage/production yes. For gas the Power meter only ‘queries’ the Gas meter every hour. So the numbers are not accurate enough to give nice graphs like the power meter does. It would be possible to add a derivative of the Gas meter reading to create a hourly usage as mentioned.
I didn’t expect the parse error to disappear, I thinks all the \x's are messing stuff up. Don’t know what they are, maybe invalidly encoded characters? Could be that sometimes the serial line gets misaligned. Which settings are you currently using?
Looking at the timestamp of the logging it appears the reading is now continuing after error, right?
Great job. Could you be so nice to try the component out with your setup, just install this file in your /custom_components folder: https://raw.githubusercontent.com/aequitas/home-assistant/dsmr/homeassistant/components/sensor/dsmr.py
I would like to know if it works for as many people before having it merged.
No @aequitas it did not continue any reading. I did try a cu -l and see the telegrams coming in fine. I also tried a full reboot but with similar results.
Just to check; did you change anything in the needs for the configuration.yaml? Mine reads the following under sensor:
- name: dsmr
platform: dsmr
port: /dev/ttyUSB-P1
dsmr_version: 4
Configuration looks fine. Do the settings that work for you with other program match these: https://github.com/ndokter/dsmr_parser/blob/master/dsmr_parser/serial.py#L23-L31 ?
Maybe it does not continue reading because it cannot assemble any valid telegram anymore. Does it at least keep printing those dsmr_parser.protocol: failed to parse telegram error lines every now and then (probably not every 10 seconds) without having to restart hass? In that case the error handling would work.
I decided to start from, more or less, scratch. I removed all dependencies, updated the raspberry, upgraded Home Assistant, checked if it all worked OK and downloaded the dsmr.py file and… All is working flawlessly now I don’t know which component was causing the issues but I have been running like this for a few hours now without a stoppage of the signal and without any errors in the logs!
Good to hear. Maybe there was a a file stuck that never got updated. Let me know when you do experience problems again, then we will do a chat session on the HA gitter and should solve your problems in no time.
In the mean time the component has been accepted by HA maintainers and should be included in the next release.
I just installed the DMR branch I pulled from your repo, and while some of the reading works nice, I get a few of these parsing errors too:
ERROR:dsmr_parser.protocol:failed to parse telegram
Traceback (most recent call last):
File "/home/pi/.homeassistant/deps/dsmr_parser/protocol.py", line 82, in handle_lines
parsed_telegram = self.telegram_parser.parse(self.telegram)
File "/home/pi/.homeassistant/deps/dsmr_parser/parsers.py", line 32, in parse
obis_reference, dsmr_object = self.parse_line(line_value.strip())
File "/home/pi/.homeassistant/deps/dsmr_parser/parsers.py", line 47, in parse_line
return obis_reference, parser.parse(line_value)
File "/home/pi/.homeassistant/deps/dsmr_parser/parsers.py", line 135, in parse
return CosemObject(self._parse(line))
File "/home/pi/.homeassistant/deps/dsmr_parser/parsers.py", line 84, in _parse
raise ParseError("Invalid '%s' line for '%s'", line, self)
dsmr_parser.exceptions.ParseError: ("Invalid '%s' line for '%s'", '1.8.1(000757.633*kWh)\r:1.8.2(000866.884*1-0:2.8.1(000000.000*kWh)', <dsmr_parser.parsers.CosemParser object at 0xb31e8eb0>)
and:
dsmr_parser.exceptions.ParseError: ("Invalid '%s' line for '%s'", '1.0(MYMETERNUMBERHERE-1:24.2.1(161123230000W)(00598./KFM5KAIFA-METER', <dsmr_parser.parsers.MBusParser object at 0xb31ec790>)
and
dsmr_parser.exceptions.ParseError: ("Invalid '%s' line for '%s'", ':2.8.1(000000.000*kWh)1-0:2.8.2(0-0:96.14.0(0001)', <dsmr_parser.parsers.CosemParser object at 0xb31e8f10>)
I used config:
- name: dsmr
platform: dsmr
port: /dev/ttyUSB0
dsmr_version: 4
Could this be due to an invalidly selected firmware? How can I find out what firmware I have?
When I installed your dsmr console, I was able to read the data using both:
dsmr_console -v --device /dev/ttyUSB0 --version 4 -p N -b 8
but also with
dsmr_console -v --device /dev/ttyUSB0 --version 4
(which resulted in even parity and a bytesize of 7)
However after installing your update all is well, and no error messages are displayed!
@Atreyu which smart meter do you have? In the user manual it should describe the DSMR version and port settings.
I now ran several hours with the new DSMR script and everything looks fine. If I might already propose some feature requests (I know: beggars can’t be choosers) my wish list would be:
- Implementation of error correction (as in DSMR4 specs)
- Recalculation of the total gas value -> hourly usage
- kindergarten.nl integration (script for this can be found at https://github.com/dennissiemensma/dsmr-reader)
But already: hiphip hooray for the great work of @aequitas!
I’m a little confused, in the beginning of your message you say it has errors, But the last line says it works now. Can I conclude you are not having parsing issues atm?
Also the code got merged into the HA dev branch. So no need fetching it from my repository anymore :).