Docker ozwd has become so slow it's not opening the VNC browser and CPU 100+%

I have had enough with VeraPlus so 2 days ago I started unpairing my z wave devices and pairing them with a zwave.me usb stick, ozwd and home assistant core each running in their own container on an RPi4. All was going well until the VNC browser got slower and slower and has now stopped connecting. I am up to 55 devices, mostly Fibaro, some POPP. Docker stats show the CPU is running at 99.7 to 100.76%, Mem usage/limit 0B/0B, Mem % is 0.00%, NET I/O is 5.03MB/45.1MB.
I have restarted the container, no change.
Any advice gratefully received!

Try posting your OS and docker info. Might also help to know which docker image you’re running and also your docker-compose or command being used to run the container.

Thank you for your advice @M0E-lnx. I will try to give you the details - please let me know how I can get the info you want if it is not included.

The docker images I have loaded are:

> REPOSITORY                                 TAG                 IMAGE ID            CREATED             SIZE
> openzwave/ozwdaemon                        allinone-latest     f24c1eefbfd6        8 days ago          501MB
> homeassistant/raspberrypi4-homeassistant   stable              efa7144649f3        12 days ago         1.06GB
> homeassistant/raspberrypi4-homeassistant   <none>              86fafb7532b1        2 weeks ago         1.06GB
> gadget1999/rpi-samba                       latest              51acff1386ea        2 weeks ago         46.6MB
> hello-world                                latest              851163c78e4a        9 months ago        4.85kB

The command I use to run the ozwd container is:

docker run -it --security-opt seccomp=unconfined --name="ozwd" --device=/dev/ttyACM0 -p 1983:1983 -p 5901:5901 -p 7800:7800 -v /home/paul/ozwd:/opt/ozw/config -e MQTT_SERVER="192.168.1.38" -e MQTT_USERNAME="private" -e MQTT_PASSWORD="private" -e USB_PATH=/dev/ttyACM0 -e OZW_NETWORK_KEY="private" --restart always openzwave/ozwdaemon:allinone-latest

The operating system is:

PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster

This is the end of the current log in ozwadmin:

2020-10-28 21:17:56.797336557  [20201028 21:17:56.797 UTC] [ozw.manager] [debug]: QTOZWOptions Ready!
2020-10-28 21:18:00.766448735  QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
2020-10-28 21:18:00.768833591  ozwadmin: Settings Path is at  "/root/.config/OpenZWave/ozw-admin.conf"
2020-10-28 21:18:00.769042551  ozwadmin: Checking  "/opt/ozw/config/"  for manufacturer_specific.xml
2020-10-28 21:18:00.769126847  ozwadmin: Checking  "/opt/ozw/config/"  for options.xml
2020-10-28 21:18:00.769162105  ozwadmin: DBPath:  "/opt/ozw/config/"
2020-10-28 21:18:00.769201790  ozwadmin: userPath:  "/opt/ozw/config/"
2020-10-28 21:18:00.906035886  ozwadmin: Settings Path:  "/root/.config/OpenZWave/ozw-admin.conf"
2020-10-28 21:18:00.906096774  ozwadmin: Using Saved Network Cache Size: 10000
2020-10-28 21:18:00.907384127  [20201028 21:18:00.907 UTC] [ozwadmin] [info]: OZWCore Initilized QTOpenZwave(0x1220bb0) QTOZWManager(0x1262878)
2020-10-28 21:18:00.971867349  [20201028 21:18:00.971 UTC] [default] [warning]: QVncServer created on port 5901
2020-10-28 21:18:07.760033989  [20201028 21:18:07.759 UTC] [ozw.manager] [debug]: QTOZWOptions Ready!
2020-10-28 22:11:57.185682446  [20201028 22:11:57.185 UTC] [ozwadmin] [debug]: Doing Remote Connection: QUrl("ws://localhost:1983")
2020-10-28 22:11:57.188277041  [20201028 22:11:57.188 UTC] [ozw.options] [debug]: Nothing to do till we connect
2020-10-28 22:11:57.206616422  [20201028 22:11:57.206 UTC] [ozw.manager] [debug]: Attempting Connection to  QUrl("ws://localhost:1983")
2020-10-28 22:11:57.207004509  [20201028 22:11:57.206 UTC] [ozw.manager] [info]: WebSocket Client State Changed:  QAbstractSocket::ConnectingState
2020-10-28 22:11:57.207400355  [20201028 22:11:57.207 UTC] [ozwadmin] [debug]: Remote Connection Status:  QTOZWManager::ConnectingState QAbstractSocket::UnknownSocketError
2020-10-28 22:11:57.207603038  [20201028 22:11:57.207 UTC] [ozwadmin] [debug]: QTOZWManager::ConnectingState QAbstractSocket::UnknownSocketError

There is a known issue when ozw disconnects from MQTT and times out that causes it to go to 100%. Do you see anything like that in the logs? Try restarting MQTT and then the ozw docker.

The ozwdaemon docker image has been updated quite a lot recently with some broken builds and as far as I know has had issues since version 150, so I think it’d be wise to fire up 150 to see if that helps anything.

Thank you @blhoward2 and again @RickKramer for your advice. I’m at work now but when I get home in about 10 hours I will give both a try. I am very grateful for your inputs - it gives me hope I won’t have to spend another three days re-pairing all the z wave devices!

If you check the ozwdaemon log (not ozw-admin), you would be able to confirm if the problem is the linked qt-openwave issue 140.

Restarting the Mosquitto MQTT server (it’s on a different Pi) made no difference. I’ll find version 150 of OZW and try that in the next couple of days.
Here is the current log from oswdaemon:

2020-10-29 20:33:38.563290784  [20201029 20:33:38.562 UTC] [ozw.daemon] [debug]: Checking  "/opt/ozw/config"  for manuf$
2020-10-29 20:33:38.591602881  [20201029 20:33:38.563 UTC] [ozw.daemon] [debug]: Checking  "/opt/ozw/config"  for optio$
2020-10-29 20:33:38.591619455  [20201029 20:33:38.563 UTC] [ozw.daemon] [info]: DBPath:  "/opt/ozw/config/"
2020-10-29 20:33:38.591629659  [20201029 20:33:38.563 UTC] [ozw.daemon] [info]: userPath:  "/opt/ozw/config/"
2020-10-29 20:33:38.591637103  [20201029 20:33:38.564 UTC] [ozw.daemon] [info]: Network Key Specified in Enviroment is $
2020-10-29 20:33:38.591643936  [20201029 20:33:38.564 UTC] [ozw.daemon] [info]: Didn't Find Network Key File. Skipping
2020-10-29 20:33:38.591650362  [20201029 20:33:38.564 UTC] [ozw.daemon] [info]: We Have what appears to be a valid Netw$
2020-10-29 20:33:38.591656566  [20201029 20:33:38.565 UTC] [ozw.daemon] [info]: Didn't Find Auth Key File. Skipping
2020-10-29 20:33:38.591662473  [20201029 20:33:38.565 UTC] [ozw.manager] [debug]: Database Path:  "/opt/ozw/config/"  U$
2020-10-29 20:33:38.591668528  [20201029 20:33:38.566 UTC] [ozw.manager] [debug]: Models Created
2020-10-29 20:33:38.591674102  [20201029 20:33:38.568 UTC] [qt.remoteobjects] [warning]:  Tried to register QRemoteObje$
2020-10-29 20:33:38.591680010  [20201029 20:33:38.568 UTC] [ozw.manager] [debug]: QTOZWManager Ready!
2020-10-29 20:33:38.591685491  [20201029 20:33:38.568 UTC] [ozw.daemon] [debug]: Ready
2020-10-29 20:33:38.591691158  [20201029 20:33:38.568 UTC] [ozw.manager] [debug]: QTOZWLog Ready!
2020-10-29 20:33:38.591696898  [20201029 20:33:38.568 UTC] [ozw.manager] [debug]: QTOZWOptions Ready!
2020-10-29 20:33:38.591702435  [20201029 20:33:38.568 UTC] [ozw.daemon] [info]: Staring  "ozwdaemon"  Version:  "0.1.17$
2020-10-29 20:33:38.591708305  [20201029 20:33:38.569 UTC] [ozw.daemon] [info]: OpenZWave Version:  "1.6.1392"
2020-10-29 20:33:38.591714676  [20201029 20:33:38.569 UTC] [ozw.daemon] [info]: QT-OpenZWave Version:  "1.2.0"
2020-10-29 20:33:38.591720324  [20201029 20:33:38.569 UTC] [ozw.daemon] [info]: QT Version:  5.12.9
2020-10-29 20:33:38.591725990  [20201029 20:33:38.569 UTC] [default] [info]: Docker MQTT_PASSWORD Secret Missing....
2020-10-29 20:33:38.591731823  [20201029 20:33:38.569 UTC] [default] [info]: Using MQTT_PASSWORD from Enviroment
2020-10-29 20:33:38.591737434  [20201029 20:33:38.569 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "ping"
2020-10-29 20:33:38.591742731  [20201029 20:33:38.569 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "open"
2020-10-29 20:33:38.591748490  [20201029 20:33:38.569 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "close"
2020-10-29 20:33:38.591754119  [20201029 20:33:38.569 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "refreshno$
2020-10-29 20:33:38.591759878  [20201029 20:33:38.570 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "requestno$
2020-10-29 20:33:38.591765601  [20201029 20:33:38.570 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "requestno$
2020-10-29 20:33:38.591771175  [20201029 20:33:38.570 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "requestco$
2020-10-29 20:33:38.591776971  [20201029 20:33:38.570 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "requestal$
2020-10-29 20:33:38.591782785  [20201029 20:33:38.570 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "softreset$
2020-10-29 20:33:38.591788230  [20201029 20:33:38.570 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "hardreset$
2020-10-29 20:33:38.591793582  [20201029 20:33:38.570 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "cancelcon$
2020-10-29 20:33:38.591799211  [20201029 20:33:38.570 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "testnetwo$
2020-10-29 20:33:38.591834377  [20201029 20:33:38.570 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "testnetwo$
2020-10-29 20:33:38.591840729  [20201029 20:33:38.570 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "healnetwo$
2020-10-29 20:33:38.591846451  [20201029 20:33:38.570 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "healnetwo$
2020-10-29 20:33:38.591852099  [20201029 20:33:38.571 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "addnode"
2020-10-29 20:33:38.591858006  [20201029 20:33:38.571 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "removenod$
2020-10-29 20:33:38.591863543  [20201029 20:33:38.571 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "removefai$
2020-10-29 20:33:38.591869154  [20201029 20:33:38.571 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "hasnodefa$
2020-10-29 20:33:38.591874506  [20201029 20:33:38.571 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "requestno$
2020-10-29 20:33:38.591880061  [20201029 20:33:38.571 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "assignret$
2020-10-29 20:33:38.591885765  [20201029 20:33:38.571 UTC] [ozw.mqtt.commands] [debug]: Registering Command  "deleteall$

Is “Docker MQTT_PASSWORD Secret Missing” relevant? I don’t know what the password secret is.
Thank you for your advice.

Secrets are for Docker Swarm, you can ignore that. The logs don’t show anything useful, it’s just the beginning of the process starting up. Is that all there is?

Try the following command to fire up 150:

docker run -it --security-opt seccomp=unconfined --name="ozw-aio150" --device=/dev/ttyACM0 -p 1983:1983 -p 5901:5901 -p 7800:7800 -v /home/paul/ozwd:/opt/ozw/config -e MQTT_SERVER="192.168.1.38" -e MQTT_USERNAME="private" -e MQTT_PASSWORD="private" -e USB_PATH=/dev/ttyACM0 -e OZW_NETWORK_KEY="private" openzwave/ozwdaemon:allinone-amd64-150 --restart always

Oh and make sure the other container isn’t running. If version 150 works for you ensure that after a reboot they don’t both start :slight_smile:

@RickKramer you have solved the problem! Thank you for saving me hours of angst having to rebuild my zwave network (and then bumping up against the same problem).
I had to tweak your docker run command because I am running the RPi in arm32 mode (whatever that means). So I changed
openzwave/ozwdaemon:allinone-amd64-150
to

openzwave/ozwdaemon:allinone-build-150

and it’s up and running!
Thank you so much for your perseverance. How did you know about this please? And is there a way of knowing when the latest version has this issue resolved?

I see you also solved a related problem about the cache size of the VNC server so I’m also very grateful for solving that issue as well.

So, double kudos!!

It’s thanks to other people’s comments really, like @freshcoast
I’m just the messenger mostly :wink:
And having faced the same issues also helps hehe. Honestly, unsure of when there will be a good build, I bet the developers will let us know or somebody that bit the bullet

Thank you very much @freshcoast as well. I am very grateful for your help.