Python external library or module include queston

Third party integrations AppDaemon
1

I’m still having this problem and need some help in how to do this. I am putting it here in AppDaemon because it is code that needs to be AppDaemon aware.

I have some different functions that if I were in another language like C I would call a utility library that I use in all of my apps. Today I have a template file that has these functions in it, that I literally copy to a new app and start writing from there. The problem is, if I need to make a change in one of those functions, I have to edit each of my apps to do that. This is the reason we have external libraries.

Lets say I have a function called expand_group that takes as arguements (self, groupname) It uses get_state and writes to the log file using self.log. Both standard AD functions. When I have tried doing this in the past, I usually end up not being able to call the function at all, or not being able to reference the AD functions from within it.

Can someone give me an example of how I should include a single file with my utility functions in it?

2

so you mean you want to use code you have written in an app and the use it in all other apps?

make a function from the code in that app and then you can use:

    yourfunctions = self.get_app("thetittlefromtheappintheconfig")
    yourfunctions.yourfunction()

i hope thats what you are looking for.

3

From the python perspective, I think what you want is to import the file of utilities
https://docs.python.org/3/reference/import.html

But Rene’s suggestion would work just as well, as far as I can see.

4

@turboc I have an app that I call, quite literally, utils. When I get home tonight, I’ll share this with you. I believe this is exactly what you’re looking for.

The general logic of it goes…

##utils.py

import re
from datetime import datetime, timdelta

import appdaemon.appapi as appapi

#
# Utility Functions are found here, so they need not be duplicated
#    in all other apps.
#

class utils(appapi.AppDaemon): #it's ok to break convention, with good reason

    def initialize(self):
        pass

    def five_minutes_from_now(self):
        return datetime.now() + timedelta(minutes=5)

    def get_all_related_entities(self, entity_type):
        """ Compiles a RegEx for entity_type """
        re_entitytype = re.compile(".*({}).*".format(entity_type))
        entities = [entity for entity in self.get_state() if re_entity.match(entity)
                                                          if 'group' not in entity]
        return entities

some_app.py

from datetime import datetime, timedelta
import appdaemon.appapi as appapi

#
# Example App
#

class SomeApp(appapi.AppDaemon):

    def initialize(self):
        self.utils = self.get_app('utils')

    def set_timer(self, minutes):
        if not minutes:
            future_time = self.utils.five_minutes_from_now()
        else:
            future_time = datetime.now() + timedelta(minutes=minutes)

##appdaemon.cfg

the normal junk...

# Apps

[utils]
module = utils
class = utils

[some_app]
module = some_app
class = SomeApp
dependencies = util

Does this make sense to you? So whenever you update utils.py, all apps that depend on it will update themselves as well. It is a way to cascade your changes through all your dependent apps so you need to only change 1 file.

Hope this helps,

  • SN
1 Like
5

That’s just about there. The problem is that I’m overriding some of the AD commands like log (sorry but typing in module and line every time I want to put in a log statement, just is to much typing. ) and when I do that, it opens a whole other ballgame. Or if it doesn’t I’m just not seeing how to do it. By using another app like this, self appears to relate to the utils app instead of the calling app. Which isn’t what I need.

I really need a simple C include statement :slight_smile:

6

Instead of overriding, why don’t you subclass AppDaemon (located in appapi.py)? Or if you really didn’t like the current logging feature, create your own customer logger? Then you can import your file just as you would like any other import. If all you’re really looking to do is customize the logger, that should be really simple - but you’re right, not very practical as an app.

– if that’s not what you’re looking to do, please be more specific. As your description from the beginning can be solved by how I originally answered.

7

subclassing is probably the correct way to do it. When I override something it’s usually with the intent of a small tweak that still uses the core functionality of the original function. It’s just more of a sledge hammer approach than I want to take. I may just put together my own little pre-processor that inserts the code for me. It’s probably more work that subclassing appapi and I may grow tired of it and quit using it. I don’t know how I’m going to tackle it. I may just keep copying a template in every time and keep making changes in all my apps anytime I change or add something to the file. It’s just the frustrations of a C developer in a python world. :slight_smile:
Thanks

8

I would still highly suggest you subclass AppDaemon. Look at AppDaemon._sub_stack() and do something similar, incorporating inspect.get_frame_info().lineno. and inspect.get_frame_info().lineno. Then you can import your subclass instead of AppDaemon.

More on inspect: https://docs.python.org/3.5/library/inspect.html#inspect.getmembers
And get_frame_info(): https://docs.python.org/3.5/library/inspect.html#inspect.getframeinfo

9

dont try to change python into C, but try to learn what python can give you that C couldnt give you :wink:

10

If you just want to add something to the original function of an inherited class then you can call the original function directly using super()

If you want to do more then you have to replicate the entire function.

11

I had been meaning to set up a class for your log function ever since I saw it. This post inspired me to go ahead and do it as an example.

$ cat apps/logged_app.py 

import appdaemon.appapi as appapi

class LoggedApp(appapi.AppDaemon):
    def log(self,message,level="INFO"):
        levels = {
                "CRITICAL": 50,
                "ERROR": 40,
                "WARNING": 30,
                "INFO": 20,
                "DEBUG": 10,
                "NOTSET": 0
                }
        try:
            if levels[level]>=levels[self.args["LOGLEVEL"]]:
                super().log("{} - {}".format(level,message))
        except KeyError:
            super().log("{}".format(message),level)

and to use it I do

$ cat apps/motion_lights.py 

import logged_app as appapi
import datetime

#
# App to turn on lights when motion detected, and to turn
# them off when motion stopped after a delay
#

class MotionLights(appapi.LoggedApp):

    def initialize(self):
        self.log("initalize", level="DEBUG")
12

I guess what concerns me about doing this is the following.

  1. are we bypassing AD and implementing things that might be better included in AD.
  2. are we making it so it’s harder for us to help each other because now self.log isn’t the same for all of us (for example).
  3. it makes it more difficult for us to share out work with each other because we all have different classes that we use to override AD. So let’s say you right a really cool turn_on function override, and I want to implement it, but I find that it’s using other functions that you have in your override class. I wonder if we aren’t in some ways hurting ourselves by doing this.
13

you can always personalize your own environment.
and you can share those parts.

if i write some general function (like my soundfunction) i use that in other apps too.
off course if i then share an app that depends on my soundfunction, i should also give the soundfunction with it.

we cant expect that every personalisation we would like to have would be implemented in appdaemon, others might dislike it, or andrew might see a disadvantage from it and decide against it.

i use self.soundlog(“any log entry”) a lot. but i cant expect it to be implemented. if andrew at some point decides that its usefull to implement such a thing then he can. but it doesnt stop me from using it if he doesnt :wink:

if i share an app with soundlog in it, i most likely rewrite the soundlog to log.

14

Maybe, but I suppose that is the nature of open source projects. Everyone’s needs are different. :slight_smile:

[quote=“turboc, post:12, topic:12593”]are we making it so it’s harder for us to help each other because now self.log isn’t the same for all of us (for example).

it makes it more difficult for us to share out work with each other because we all have different classes that we use to override AD. So let’s say you right a really cool turn_on function override, and I want to implement it, but I find that it’s using other functions that you have in your override class. I wonder if we aren’t in some ways hurting ourselves by doing this.[/quote]

I don’t see that customization makes anything more difficult. At the end of the day, it is all still Python. As long as you post the proper amount of code and ask the right questions. We are all tinkerers.

15

I guess I’m used to work where you don’t modify the source or the vendor won’t support it anymore. It’s really frustrating when the vendor says, take out your custom patch and we’ll support it. Other wise, you touched it you fix it. LOL

16

creating an app that overrides functions isnt modifiing the source :wink:
im glad that GP showed me the way to do that, because i am really someone that just changes the original code if i feel the need for it :wink:

yeah i know that i cant expect the same support if i do that, but thats the price i pay for that :wink:

17

@turboc @ReneTode

Playing off of this … I believe what you’re looking for is to have the line number and module all in the log entry, correct? This is how I would do it which I believe solves your problem. Granted, @yawor seems to have heard yours and others issues and has already implemented a PR to solve this. :slight_smile:

Regardless, hereyougo~

###appdaemon.cfg

[custom_logger]
module = custom_logger
class = CustomLogger

[utils]
module = utils
class = utils
dependencies = custom_logger

###custom_logger.py

import appdaemon.appapi as appapi
import appdaemon.homeassistant as ha
import datetime
import inspect
import re

class CustomLogger(appapi.AppDaemon):

    def initialize(self):
        pass

    def log(self, msg, level="INFO"):
        self._log(self._logger, level, msg, self.name)

    def error(self, msg, level="WARNING"):
        self._log(self._error, level, msg, self.name)

    def _log(self, logger, level, msg, name=""):

        levels = {
            "CRITICAL": 50,
            "ERROR": 40,
            "WARNING": 30,
            "INFO": 20,
            "DEBUG": 10,
            "NOTSET": 0
        }

        # stack returns list of FrameInfo(frame, filename, lineno, function, code_context, index)
        # app is 2 frames up == stack[2]
        stack = inspect.stack()

        # RegEx to extract name of app from filename
        re_name = re.compile('.*\/(.*).py')

        if ha.conf.realtime:
            timestamp = datetime.datetime.now()
        else:
            timestamp = ha.get_now()

        message_format = ('[{t}] {lvl} {name}.{mod} line {ln} {msg}'
                          .format(t=timestamp,
                                  lvl=level,
                                  name=re_name.match(stack[2][1]).group(1),
                                  mod=stack[2][3],
                                  ln=stack[2][2],
                                  msg=msg))

        logger.log(levels[level], message_format)

utils.py (for example of usage)

import appdaemon.appapi as appapi
import json

class utils(appapi.AppDaemon):

    def initialize(self):
        self.custom_logger = self.get_app('custom_logger')
        self.custom_logger.log('how bow dah')

###output logfile.log

2017-02-26 13:36:36.036629 INFO Reloading Module: /home/homeassistant/.homeassistant/appdaemon/conf/apps/custom_logger.py
2017-02-26 13:36:36.045918 INFO Loading Object custom_logger using class CustomLogger from module custom_logger
2017-02-26 13:36:36.049179 INFO Reloading Module: /home/homeassistant/.homeassistant/appdaemon/conf/apps/utils.py
2017-02-26 13:36:36.056990 INFO Loading Object utils using class utils from module utils
[2017-02-26 13:36:36.066024] INFO utils.initialize line 8 how bow dah

You’ll see the last line is different. You can customize this to your heart’s content. If you’re trying to discern your logs from those of AppDaemon, you could even add some flair to the timestamp like '===> {t} {lvl} {name}.{mod} line {ln} {msg}'. Really, it’s whatever you want.

You should be aware, if you want to use this, I would highly advise you to follow AppDaemon best-practices. Let is reside as an app which you have all other apps depend on, and then get the app and assign it to self.custom_logger.

Why should you do this? Because it will help others to understand that you have a custom logger, and this can then become necessary/unnecessary when you are seeking help on a [possibly] unrelated problem.

Hope this helps,

  • SN
1 Like
18

Actually, the purpose of my post was an example of overriding a function by subclassing, and the purpose of the log function was to start logging at a certain level from a parameter of the app.

Also, I think the functionally of your post was introduced in AppDaemon recently. You can put __function__ or __line__ in the log string, and it gets replaced appropriately.

But custom loggers look interesting, and will go in my toolbox.

1 Like
19

SupahNoob
Love your utils.py idea…
question, what if you wanted to pass arguments to a object in utils, how would you do that?
I have some code to send an email and would like to dynamically pass the subject and message and leave to code to perform the send in utils.py…
Thanks

20

i use sensors for that.
my notify app could inspire you to do something like it for mail:

import appdaemon.plugins.hass.hassapi as hass


class notify(hass.Hass):

  def initialize(self):
    if not self.entity_exists("sensor.notify_message"):
      self.set_state("sensor.notify_message",state=" ")
    self.listen_state(self.prepare_notify,"sensor.notify_message")

  def prepare_notify(self, entity, attribute, old, new, kwargs):
    if new == " ":
      return
    counter = 0
    repeat = 15
    messageamount = 1
    message = new
    if "::" in new:
      messageparts = new.split("::")
      if len(messageparts) == 3:
        repeat = int(messageparts[2])
      messageamount = int(messageparts[1])
      message = messageparts[0]

    while counter < messageamount:
      self.run_in(self.send_notify,1+(counter*repeat),message=message)
      counter = counter + 1
    self.set_state("sensor.notify_message",state=" ")

  def send_notify(self,kwargs):
    self.call_service("notify/pushetta",message=kwargs["message"])
    self.call_service("notify/gmail",message=kwargs["message"])
    self.call_service("notify/handy_rene",message=kwargs["message"])
    sound = self.get_app("soundfunctions")
    sound.say(kwargs["message"],"nl","1")
    self.log(kwargs["message"])

in any app i can use self.set_state(“sensor.notify_message”, state=“any message i like to be send”)