DSMR Reader Add-on for Home Assistant

rednas · January 11, 2022, 10:31am

Hello,

I am having the same kind of problems I guess. Every minute or so the plugin restarts. I can see this in the logs.
This is the last log I could copy:

[ INFO ] DSMR release: v4.19.0

[ INFO ] Removing existing PID files...

[ INFO ] Creating log directory...

[ INFO ] Setting architecture requirements...

[ INFO ] X64 Architecture

[ INFO ] Enabling IFrame...

[ INFO ] Fixing /dev/ttyUSB* security...
chmod: /dev/ttyUSB0: Read-only file system

[ INFO ] Verifying if the DSMR web credential variables have been set...

[ INFO ] Verifying database connectivity to host: 77b2833f-timescaledb with port: 5432...

[ INFO ] Database connectivity successfully verified!

[ INFO ] Running post configuration...
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, dsmr_api, dsmr_backend, dsmr_backup, dsmr_consumption, dsmr_datalogger, dsmr_dropbox, dsmr_frontend, dsmr_influxdb, dsmr_mindergas, dsmr_mqtt, dsmr_notification, dsmr_pvoutput, dsmr_stats, dsmr_weather, sessions
Running migrations:
  No migrations to apply.
563 static files copied to '/var/www/dsmrreader/static'.
Updating password of superuser "XXXXXX"
Deactivating any other existing superusers

[ INFO ] Checking for NGINX SSL configuration...

[ INFO ] ENABLE_NGINX_SSL is disabled, nothing to see here. Continuing...

[ INFO ] Checking for HTTP AUTHENTICATION configuration...

[ INFO ] ENABLE_HTTP_AUTH is disabled, nothing to see here. Continuing...

[ INFO ] Configuring DSMR in standlone datalogger mode....

[ INFO ] Starting supervisord...

[ INFO ] Logfiles can be found at: /var/log/supervisor/*.log and /tmp/supervisord.log
2022-01-11 11:22:02,946 INFO Set uid to user 0 succeeded
2022-01-11 11:22:02,946 INFO Set uid to user 0 succeeded
2022-01-11 11:22:02,958 INFO RPC interface 'supervisor' initialized
2022-01-11 11:22:02,958 INFO RPC interface 'supervisor' initialized
2022-01-11 11:22:02,959 INFO supervisord started with pid 35
2022-01-11 11:22:02,959 INFO supervisord started with pid 35
2022-01-11 11:22:03,964 INFO spawned: 'processes' with pid 37
2022-01-11 11:22:03,964 INFO spawned: 'processes' with pid 37
2022-01-11 11:22:03,969 INFO spawned: 'dsmr_backend' with pid 38
2022-01-11 11:22:03,969 INFO spawned: 'dsmr_backend' with pid 38
2022-01-11 11:22:03,976 INFO spawned: 'dsmr_datalogger' with pid 39
2022-01-11 11:22:03,976 INFO spawned: 'dsmr_datalogger' with pid 39
2022-01-11 11:22:03,984 INFO spawned: 'dsmr_webinterface' with pid 40
2022-01-11 11:22:03,984 INFO spawned: 'dsmr_webinterface' with pid 40
2022-01-11 11:22:03,991 INFO spawned: 'nginx' with pid 41
2022-01-11 11:22:03,991 INFO spawned: 'nginx' with pid 41
[2022-01-11 11:22:04 +0100] [40] [INFO] Starting gunicorn 20.1.0
[2022-01-11 11:22:04 +0100] [40] [INFO] Listening at: unix:/var/tmp/gunicorn--dsmr_webinterface.socket (40)
[2022-01-11 11:22:04 +0100] [40] [INFO] Using worker: sync
[2022-01-11 11:22:04 +0100] [46] [INFO] Booting worker with pid: 46
2022-01-11 11:22:05,611 INFO success: processes entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-01-11 11:22:05,611 INFO success: processes entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-01-11 11:22:05,613 INFO success: dsmr_backend entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-01-11 11:22:05,613 INFO success: dsmr_backend entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-01-11 11:22:05,615 INFO success: dsmr_datalogger entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-01-11 11:22:05,615 INFO success: dsmr_datalogger entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Current logging level set to "ERROR". More information can be found here: https://dsmr-reader.readthedocs.io/en/latest/how-to/troubleshooting/enabling-debug-logging.html
Current logging level set to "ERROR". More information can be found here: https://dsmr-reader.readthedocs.io/en/latest/how-to/troubleshooting/enabling-debug-logging.html
2022-01-11 11:22:07,140 INFO success: dsmr_webinterface entered RUNNING state, process has stayed up for > than 3 seconds (startsecs)
2022-01-11 11:22:07,140 INFO success: dsmr_webinterface entered RUNNING state, process has stayed up for > than 3 seconds (startsecs)
2022-01-11 11:22:09,143 INFO success: nginx entered RUNNING state, process has stayed up for > than 5 seconds (startsecs)
2022-01-11 11:22:09,143 INFO success: nginx entered RUNNING state, process has stayed up for > than 5 seconds (startsecs)
2022-01-11 11:26:24,611 ERROR    mixins       run_once                        103 | dsmr_datalogger.management.commands.dsmr_datalogger: [!] Exception raised. Traceback (most recent call last):
  File "/dsmr/dsmr_backend/mixins.py", line 92, in run_once
    self.run(data=self.data, **options)
  File "/dsmr/dsmr_datalogger/management/commands/dsmr_datalogger.py", line 29, in run
    telegram = next(self.telegram_generator)
  File "/dsmr/dsmr_datalogger/scripts/dsmr_datalogger_api_client.py", line 50, in read_telegram
    incoming_bytes = serial_handle.read(MAX_BYTES_PER_READ)
  File "/usr/local/lib/python3.10/site-packages/serial/serialposix.py", line 595, in read
    raise SerialException(
serial.serialutil.SerialException: device reports readiness to read but returned no data (device disconnected or multiple access on port?)
2022-01-11 11:26:29,855 INFO exited: dsmr_datalogger (exit status 0; expected)
2022-01-11 11:26:29,855 INFO exited: dsmr_datalogger (exit status 0; expected)
2022-01-11 11:26:30,860 INFO spawned: 'dsmr_datalogger' with pid 47
2022-01-11 11:26:30,860 INFO spawned: 'dsmr_datalogger' with pid 47
2022-01-11 11:26:30,862 WARN received SIGQUIT indicating exit request
2022-01-11 11:26:30,862 WARN received SIGQUIT indicating exit request
2022-01-11 11:26:30,863 INFO waiting for processes, dsmr_backend, dsmr_datalogger, dsmr_webinterface, nginx to die
2022-01-11 11:26:30,863 INFO waiting for processes, dsmr_backend, dsmr_datalogger, dsmr_webinterface, nginx to die
2022-01-11 11:26:30,880 INFO stopped: nginx (exit status 0)
2022-01-11 11:26:30,880 INFO stopped: nginx (exit status 0)
[2022-01-11 11:26:30 +0100] [40] [INFO] Handling signal: term
[2022-01-11 11:26:30 +0100] [46] [INFO] Worker exiting (pid: 46)
[2022-01-11 11:26:31 +0100] [40] [INFO] Shutting down: Master
2022-01-11 11:26:32,253 INFO success: dsmr_datalogger entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-01-11 11:26:32,253 INFO success: dsmr_datalogger entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-01-11 11:26:32,255 INFO stopped: dsmr_webinterface (exit status 0)
2022-01-11 11:26:32,255 INFO stopped: dsmr_webinterface (exit status 0)
Current logging level set to "ERROR". More information can be found here: https://dsmr-reader.readthedocs.io/en/latest/how-to/troubleshooting/enabling-debug-logging.html2022-01-11 11:26:32,379 INFO stopped: dsmr_datalogger (terminated by SIGTERM)
2022-01-11 11:26:32,379 INFO stopped: dsmr_datalogger (terminated by SIGTERM)
2022-01-11 11:26:33,626 INFO stopped: dsmr_backend (exit status 0)
2022-01-11 11:26:33,626 INFO stopped: dsmr_backend (exit status 0)
2022-01-11 11:26:33,627 INFO stopped: processes (terminated by SIGTERM)
2022-01-11 11:26:33,627 INFO stopped: processes (terminated by SIGTERM)

This is my TimescaleDB config:

databases:
  - homeassistant
  - dsmrreader
timescale_enabled:
  - homeassistant
  - dsmrreader
timescaledb:
  telemetry: basic
  maxmemory: 512MB
  maxcpus: '4'
max_connections: 20
system_packages: []
init_commands: []

(A side question: does the dsmrreader database need timescale enabled?)

Does anyone know what is going on and how I can fix this??

sanderdw · January 11, 2022, 10:46am

No don’t put the db in timescale_enabled, it creates only overhead which you don’t need. Which exact hardware are you guys running?

rednas · January 11, 2022, 11:45am

Ok, thanks I removed it from the timescale_enabled as I’m not running it for long and don’t mind that my data is probably gone.

I am running HA OS 7.1 with Core 2021.12.8 on an Intel NUC with Proxmox. Do you require more information?

fxx · January 11, 2022, 9:26pm

I run HAOS 2021.12.9 on a Dell Wase 5020 with 32gb flashdrive which according to HA is about 61% full.a

rednas · January 12, 2022, 1:27pm

Maybe my problems are because I am also using the port for

- platform: dsmr

I also tried adding my MQTT settings to the DSMR reader to check whether it works in Home Assistant, but I entered something wrong. I can see in the logs that DSMR reader is again restarting a lot, but now I am unable to enter DSMR reader completely. Sometimes the login page shows up, but when I enter it probably restarts already and it shows that the page is unreachable… (I also tried removing and readding it, but it looks like it remembers the MQTT settings)
So basically I am locked out now… How do I change my MQTT settings now?

sanderdw · January 12, 2022, 2:32pm

You can’t run both this and the DSMR Reader addon as they both need the USB port. With this integration you can get the data back in HA: DSMR Reader - Home Assistant

You can try to change the MQTT settings in the database, it’s in the dsmr_mqtt_mqttbrokersettings table.

rednas · January 12, 2022, 3:09pm

Thanks. I think I got it to work.
However I don’t get the data back in HA with the link you provided. All sensors are created, but all of them show “unknown”.
MQTT log shows:

1641999673: New client connected from 172.30.32.1 as DSMR-reader (p2, c1, k60, u'mosquitto').
1641999764: Client DSMR-reader has exceeded timeout, disconnecting.

Does this matter? Do you know what I could be doing wrong?

ADDED LATER:
I also found this in the DSMR Reader settings, so there are 0 outgoing MQTT messages. Why is this? I selected all of the split topics under MQTT.

soepkip · January 14, 2022, 1:02pm

This looks interesting! Thanks for developing & maintaining this!

I am currently running a ‘native’ DSMR reader on a Rpi3.
I am planning to install HA on a Rpi3 (or 4) and integrate DSMR Reader into that…

Would I be able to restore my DSMR reader database into this HA-installed Add-on version?
(Because it seems like a Docker installed ‘in’ HA, a bit tricky maybe…)

So something like:

Get a fresh Pi, install Raspbian & update etc.
Install HA
Install DSMRR-Add On
Restore dsmr DB (how?)

nelbs · January 14, 2022, 5:59pm

Yes that’s possible. I did it through the console of the timescale DB container. See this issue

github.com/xirixiz/dsmr-reader-docker

restore backup with hassio dsmr-addon

opened 09:10PM - 04 Mar 21 UTC

closed 12:17PM - 11 Mar 21 UTC

nelbs

bug

Ik heb onlangs de Homeassistant [addon dsmr-reader ](https://github.com/sanderdw…/hassio-addons)geinstalleerd. Alles werkt naar behoren maar het herstellen van een backup van mijn oude installatie lukt me niet. Ik weet dat ik niet direct deze container gebruik maar wellicht dat iemand mij verder kan helpen. ## Setup/Architecture information Pi4 Homeassistant supervised met DSMR-reader en TimescaleDB addons ## Version of the Docker image Addon dsmr reader: 0.1.2 addon timescale DB 1.1.6 ## Configuration nvt ## Describe the bug Ik heb het volgende allemaal geprobeerd: **1)** Als ik de container probeer te stoppen met docker compose stop krijg ik de volgende melding: ``` ERROR: Can't find a suitable configuration file in this directory or any parent. Are you in the right directory? Supported filenames: docker-compose.yml, docker-compose.yaml ``` ergens is dit logisch aangezien ik de container niet met compose heb opgezet... **2)** als ik de container via portainer stop en vervolgens de backup wil terug zetten met het volgende commando: `docker exec -t addon_0493853b_dsmr_reader createdb -O dsmrreader dsmrreader -U dsmrreader` krijg ik de volgende melding: `Error response from daemon: Container ..... is not running` **3)** Als ik hetzelfde commando geef als de container draait dan krijg ik geen melding maar van een backup is ook geen sprake. ## Debug log ```text Add your logs here. ```

sanderdw · January 17, 2022, 2:42pm

2022-01-17: 0.7.3 Image update and debug option

sanderdw · January 17, 2022, 2:45pm

First you need to check if DSMR and MQTT are configured correctly. An app like http://mqtt-explorer.com/ could help you with this.

rednas · January 17, 2022, 2:49pm

Is there a possibilty you could add the option to add DSMR Reader to the side bar like other addons have as well?

nelbs · January 17, 2022, 8:36pm

If you are accessiing HA over http you can use a iframe Panel - Home Assistant

rednas · January 18, 2022, 8:38am

I am accessing HA over https, so that’s not an option unfortunately.
Is that also a problem when adding it to the side bar by adding it to the addon, or not?

sanderdw · January 18, 2022, 4:29pm

Ingress would be the best way to do this but i’m struggling a bit how this should be set-up, see my last comment in this issue: Reverse proxy support · Issue #24 · sanderdw/hassio-addons · GitHub

sanderdw · January 18, 2022, 4:31pm

2022-01-18: 0.7.4 Reverting again to solve CPU and backup permission errors

woutercoppens · January 18, 2022, 9:01pm

If I understand there are 3 possibilities to get the dsmr data:

via the integration: https://www.home-assistant.io/integrations/dsmr
via dsmr_reader: https://www.home-assistant.io/integrations/dsmr_reader/
via this addon: https://github.com/sanderdw/hassio-addons

What is the difference between these? Advantages/disadvantages?

sanderdw · January 18, 2022, 10:20pm

The first one is indeed a separate solution, which works on it’s own. Downside is that you don’t get all the features the DSMR Reader is providing: About DSMR-reader — DSMR-reader v4 documentation

The second option is in fact an addition to this addon (see step 12 in hassio-addons/dsmr_reader at master · sanderdw/hassio-addons · GitHub). This addon by itself don’t integrate any ‘sensors’ into HA. So use them in a combination .

rednas · January 19, 2022, 7:18am

Ok, can’t help you with that unfortunately, but I hope you can find a solution soon!

cece · February 1, 2022, 2:11pm

Hi Rednas, could you explain what you did to solve so that the pluging doesn’t restart everytime? Mine is also crashing al the time and the webinterface is not reachable.