Thread errors

Third party integrations AppDaemon
1

Currently getting these warnings in the logs:

2018-09-07 11:25:27.809808 WARNING AppDaemon: Queue size is 140, suspect thread starvation

2018-09-07 11:25:27.810155 INFO AppDaemon: --------------------------------------------------

2018-09-07 11:25:27.810332 INFO AppDaemon: Threads

2018-09-07 11:25:27.810529 INFO AppDaemon: --------------------------------------------------

2018-09-07 11:25:27.810739 INFO AppDaemon: Currently busy threads: 0

2018-09-07 11:25:27.810966 INFO AppDaemon: Most used threads: 1 at 2018-09-07 11:23:13

2018-09-07 11:25:27.811157 INFO AppDaemon: Last activity: 2018-09-07 11:23:13

2018-09-07 11:25:27.811343 INFO AppDaemon: --------------------------------------------------

2018-09-07 11:25:27.811822 INFO AppDaemon: thread-1 - current callback: idle since 2018-09-07 11:23:13, alive: False

2018-09-07 11:25:27.812069 INFO AppDaemon: thread-2 - current callback: idle since 1970-01-01 08:00:00, alive: False

2018-09-07 11:25:27.812278 INFO AppDaemon: thread-3 - current callback: idle since 1970-01-01 08:00:00, alive: False

2018-09-07 11:25:27.812488 INFO AppDaemon: thread-4 - current callback: idle since 1970-01-01 08:00:00, alive: False

2018-09-07 11:25:27.812679 INFO AppDaemon: thread-5 - current callback: idle since 1970-01-01 08:00:00, alive: False

2018-09-07 11:25:27.812905 INFO AppDaemon: thread-6 - current callback: idle since 1970-01-01 08:00:00, alive: False

2018-09-07 11:25:27.813108 INFO AppDaemon: thread-7 - current callback: idle since 1970-01-01 08:00:00, alive: False

2018-09-07 11:25:27.813306 INFO AppDaemon: thread-8 - current callback: idle since 1970-01-01 08:00:00, alive: False

2018-09-07 11:25:27.813503 INFO AppDaemon: thread-9 - current callback: idle since 1970-01-01 08:00:00, alive: False

2018-09-07 11:25:27.813725 INFO AppDaemon: thread-10 - current callback: idle since 1970-01-01 08:00:00, alive: False

AND have narrowed it down to this rogue app - any clues?

from base import Base
import datetime
import time
import globals

## Notify of potential intruders

class IntruderAlert(Base):

  def initialize(self) -> None:
    super().initialize()
    if "sensor" in self.args:
      for sensor in self.split_device_list(self.args["sensor"]):
        self.listen_state(self.intruder, sensor)
        self._tts_device = "media_player.living, media_player.bedroom"

  def intruder(self, entity, attribute, old, new, kwargs):
    if new == "on":
      self.call_service("notify/ios_grants_iphone", title = "Intruder Alert", message = "Intruder Alert in the {}".format(self.args["room"]))
      self.log("INTRUDER ALERT IN THE {}".format(self.args["room"]))
      self.tts_manager.set_volume_level('0.4', media_player=self._tts_device)
      self.tts_manager.speak("You are not authorised to be here - police have been notified", media_player = self._tts_device)```
2

you have a strange way of using appdaemon.

from base import Base
class IntruderAlert(Base):

is not the normal way apps are setup. (and that can cause serious trouble)
another possibility is that you motion sensors send to many changes to fast.

3

Hi Rene, Just a new player to the AD space and experimenting with code that is available out there at the moment. I think I narrowed down in the end to another app using “self.run_hourly” line.

4

the base structure from appdaemon should always be:

import appdaemon.plugins.hass.hassapi as hass
class forum(hass.Hass):
  def initialize(self):

to me it seems that you have 1 app like that and you import that as a class in your other “apps”
the problem you create with that is that you run everything trough 1 thread.

so my advice is to use the “normal” appdaemon structure and functions to avoid big trouble.

1 Like
5

I use base classes everywhere, this won’t hurt anything. His class using his base class is setup properly with super().initialize(). Seems like he just comes from a true object oriented thinking.

6

could be.
did you go through the AD code to see if there is nothing that can go wrong with that kind of working?

at least it isnt the proposed/supported way of using AD, and it goes against the docs.
thats why i cant garantee that there are no unexpected problem coming up and its very hard to support.

7

I’m running into this same problem. Did you find a resolution?

8

I haven’t looked at the code, but it still shouldn’t harm it. Making a class that uses hass.Hass as it’s base will just end up saving you steps if you tend to repeat the same lines of code for every app deamon app that you write.

Example, If you tend to print out your config to the logs for every app daemon app, do you want to write self.log(self.args, level='DEBUG') in every app or would you rather create a base class that has that?

class wrappedhass(hass.Hass):
    def initialize(self):
        self.log("blah", level='DEBUG')
        self.log(self.args, level='DEBUG')
        self.log('end blah', level='DEBUG')

then all your classes would just be the following code and they would always output those 3 lines. Everything else would be identical behavior and you’d treat your wrappedhass object as the hass.Hass object.

class mystuff(wrappedhass):
   def initialize(self):
       super().initialize()
9

in that case functions like get_app and global settings would be completely obsolete.

appdaemon is multithreaded.
and i am not sure if the way you describe will use the threadsystem in AD the way it is supposed to.

ill ask andrew about it, to make sure i support things the right way :wink:

i forsee at least 1 thing that should be noticed.
the super app needs to be marked as global and global dependencies should be in the yaml.

10

Hi everyone,
Hopefully I am posting to the correct topic. I am fairly new to AppDaemon and started off with a CANBUS reader and interpreter.
However after about 10 hours of working perfectly the following warning and error occured in the logs and no further data is passed to the sensors:

AppDaemon log:

WARNING AppDaemon: Queue size for thread thread-0 is 112, callback is ‘process_data() in canbus_processor’ called at 2023-10-19 09:59:45+02:00 - possible thread starvation

The CANBUS source is a gateway which acts as a TCP server sending frames in multiple of 26 hex values (ususally 1 to 6 frames every few seconds). These frames are written into a buffer and sequentially split up into seperate frames of 26 values each which are then analyzed for their ID and value. The values are then mapped to sensor names.

Here’s the code:

import appdaemon.plugins.hass.hassapi as hass
import socket
from datetime import datetime

class CANBusDataProcessor(hass.Hass):

    def initialize(self):
        self.data_buffer = ""  # Initialize an empty data buffer as a string
        self.tcp_host = "192.168.178.201" 
        self.tcp_port = 8881 
        self.tcp_socket = None
        self.connect_to_tcp() 
        self.run_every(self.process_data, datetime.now(), 30)

        self.parameter_mapping = {
            '0001': 'fehlermeldung',
            '000c': 'aussentemperatur',
            '000e': 'speichertemperatur',
            '000f': 'vorlauftemperatur',
            '0011': 'raumtemperatur',
            '0075': 'raumfeuchte',
            '001a': 'kollektortemperatur',
            '0112': 'programm',
            '0121': 'wochentag',
            '0122': 'tag',
            '0123': 'monat',
            '0124': 'jahr',
            '0125': 'stunde',
            '0126': 'minute',
            '0176': 'betriebsstatus',
            '019a': 'softwareversion'
        }

    def connect_to_tcp(self):
        try:
            self.tcp_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            self.tcp_socket.connect((self.tcp_host, self.tcp_port))
            self.log("Connected to CANBUS source.")
        except Exception as e:
            self.log(f"TCP connection error: {e}")

    def process_data(self, kwargs):
        if self.tcp_socket:
            try:
                # Receive data from the TCP connection
                data = self.tcp_socket.recv(1024)
                if data:
                    # Append received data to the buffer
                    self.data_buffer += data.hex()
                else:
                    self.log("Connection closed. Reconnecting...")
                    self.tcp_socket.close()
                    self.connect_to_tcp()  # Attempt to reconnect
            except Exception as e:
                self.log(f"Error receiving data from TCP connection: {e}")
            while len(self.data_buffer) >= 26:
            # Extract 26-character segment from the buffer
                segment = self.data_buffer[:26]

            # Extract ID from positions 16-20 and value from positions 20-24
                id_hex = segment[16:20]
                value_hex = segment[20:24]

                try:
                    # Look up the parameter name in the mapping
                    parameter_name = self.parameter_mapping.get(id_hex, "unknown_parameter")

                    # Construct sensor names and set their states
                    id_sensor_name = f'sensor.{parameter_name}_id'
                    value_sensor_name = f'sensor.{parameter_name}_can_value'

                    self.set_state(id_sensor_name, state=id_hex)
                    self.set_state(value_sensor_name, state=value_hex)

                except ValueError as e:
                    self.log(f"Error converting hexadecimal data to decimal: {e}")

                # Remove the processed data from the buffer
                self.data_buffer = self.data_buffer[26:]

    def terminate(self):
        if self.tcp_socket:
            self.tcp_socket.close()
            self.log("TCP connection closed")

Any ideas?
Thanks

11

your app is queing.
and you got only 1 returning function (the process_data)
my guess is that self.tcp_socket at some point hangs, or takes way to long.
the other option is that you get data in that gets your while loop to go forever.

if the function isnt ending, the next time it is called it will go in a queue, and the next, and the next, …

1 Like