AppDaemon: Queue size is 80, suspect thread starvation

I have an HA / Appdaemon / Hadashboard system in development. It is essentially a lighting control system with motion sensors in rooms. The main concept (at present) is to have down lights which are responsive to motion and to time of day. They change colour temperature over the day and during sleep time they become night lights with very dull orange glow. The automatic system can be stopped by an input boolean linked to a switch in the Hadashboard.
The automatic system is controlled by a couple of hundred lines of appdaemon python code. I am a beginner with python, with the properties of the hardware (pi3, milight, v6 bridges and Aeotec z-wave multisensors). I am also a beginner when it comes to getting the end users (me and wife) satisfied with the functionality. So my code is evolving and at best is untidy.
However, there are times when it all works as intended. And there are times when the Hadashboard ignores all manual input and the behaviour of the lights becomes erratic. Fortunately I run some logging in the python code and notice in the log that some events recur several times in the one second. The auto off switch in Hadashboard can repeat 8 times in the one second and even the initialise function can recur several times in the one second. (I am only logging one or two instances of the class at any one time.) Finally I notice that the message “AppDaemon: Queue size is xx, suspect thread starvation” is occurring.
So I am doing something wrongly and would be very happy if someone could give me some pointers to potential traps. If necessary I could supply code, yamls etc.
Some help would be greatly appreciated.

if switches start to go off several times in a second then you probably created a loop somewhere.
initialise happens only when AD restarts or when files change. so if that is reloaded several times, you really have created a problem.

a loop would cause that the threads from AD are filled quite fast and then you get thread starvation.

i dont mind taking a look at your code, but if its several hundreds of lines that wouldnt be easy.
if its to complicated to get the errors out, it could be that its better to start over small.

but let see your code first, so i can see if its worth to correct what you already have.

Rene, you may have nailed the problem right there and then. When the hadashboard switch is changed from “off” to “on” I have been calling the initialise function. That has looping potential!! I will look more closely and hopefully resolve the problem.
Thanks once again for your help.

hmm that is for certain creating a problem.
an initialise should not be called from the other code.
and when you got a listen_state in the initialise (which is the best place to have them) then you start adding threads, which for itselve will start calling the callbacks, and you get a flood.
i think you are lucky that you didnt hang your device completely :wink:

Yes, that was the cause of the problem. I separated out some variable initialisations and executed these separately to avoid the callbacks in the initialize function. Seems to be working correctly now.
Your insights and experience have been invaluable for me.

1 Like