[AppDaemon] Tutorial #3 Utility Functions

Hi everybody!

Recently I gave a user some feedback who was looking to convert their YAML Automation file to the app-based Python structure of AppDaemon. It dawned on me that a lot of users who are using AppDaemon are seeking the extra flexibility that Python can afford to offer, but may not necessarily be familiar with how to write Python in the first place! Some may have a CS background, while others may not. This post is part of a tutorial series of sorts wherein I tackle a problem and show both a simple and complex way to write an AppDaemon app! The simple version will be ready-to-use right out of the box. The complex version will also run “out of the box”, but you’ll likely want to tweak it to work specifically with your own setup! I will go over more advanced concepts and style guide mentions here.

##Tutorials

  1. Tracker-Notifier - monitor devices’ on state, and receive a notification after a user-specified length of time
  2. Errorlog Notifications - have a persistent notification appear on the dash any time AppDaemon errors out
  3. Utility Functions - create general purpose functions that other apps can refer to in order to simplify your code

In the easy version, I will try to break the solution down in simple terms so that someone who knows absolutely nothing about Python can understand them. The more complex example will also have a brief description explaining general logic, but will be targeted towards those of you who have a better grasp of the fundamentals.

I hope you enjoy!


#Tutorial #3 - Utility Functions

This tutorial is going to be a bit shorter than usual, as we’re talking more about the concepts of code re-use and sandbox we have to play in is a lot bigger than our previous discussions. The sky is the limit here to be honest! Utility functions will be more geared towards your own setup and what you find you’re doing over and over again, rather than writing a single, functional automation.

— simple —

Before I get into the simple app, I want to first talk about how integral the idea of “reusing well-written code” is to Python, and programming in general. Wouldn’t it be great to write a function that does one thing perfectly, and then re-use that in all your other apps? You’re certainly already doing this, take the following example…

from datetime import timedelta

...

    def ten_minutes_from_now():
        """
        Return datetime.datetime object +10 minutes from current internal 
        AppDaemon clock
        """
        return self.datetime() + timedelta(minutes=10)

You should be able to easily tell from the description in red what the app is doing. self.datetime() is @aimc’s helper function that gives you the current time as dictated by AppDaemon. timedelta is a function from the datetime library that expresses “the difference between two date, time, or datetime instances to microsecond resolution.Both of these functions are code that someone else has written, that we are reusing and now extending to be even more useful to us! Oftentimes when you write a general purpose function like this, it’s helpful to write what we call a docstring or documentation strings. It’s usually a very short descriptor of what’s going on in this function wrapped in triple-quotes (""").

This idea of code re-use is actually one of the main ideas behind AppDaemon. Here we give an example of having written one app that takes various arguments sensor and light, where if motion is detected on sensor, then light needs to turn on. This app is reused three different times, with its parameters filled in for three different sensors: one upstairs, one downstairs, and one in the garage. This way, if you need to change some general logic for motion sensing, you only need to change it in one place.

Let’s going to take a slightly different application on the above approach and instead of developing 1 app that can be used multiple ways, we’ll create a set of utlity functions that can then be used in many of your others apps! We’ll then get the utilities app in the initialization of each of our dependent apps. But enough talking, let’s get coding! :slight_smile:

import appdaemon.appapi as appapi
from datetime import datetime, timedelta

#
# App to localize frequently used functions
#
# Args: (set these in appdaemon.cfg)
# n/a
#
# EXAMPLE appdaemon.cfg entry below...
#         ... and of how to make utils initialize before other apps 
#         get the chance to
# 
# # Apps
# 
# [utils]
# module = utils
# class = utils
#
# [hello_world]
# module = hello_world
# class = HelloWorld
# dependencies = utils
#
# class HelloWorld(appapi.AppDaemon):
#     def initialize(self):
#         self.utils = self.get_app('utils')
#         self.log('Tomorrow is {}'.format(self.utils.tomorrow())
#

class utils(appapi.AppDaemon):

    def initialize(self):
        pass

    def ten_minutes_from_now(self):
        """
        Return datetime.datetime object +10 minutes from current internal 
        AppDaemon clock
        """
        return self.datetime() + timedelta(minutes=10)

    def tomorrow(self):
        """
        Return datetime.date object +1 day from current internal AppDaemon 
        clock
        """
        return self.date() + timedelta(days=1)

    def soon(self, seconds=5):
        """
        Return datetime.datetime object some number of seconds from current
        internal AppDaemon clock. 

        Keyword arguments:
        seconds -- seconds in the future (default 5)
        """
        return self.datetime() + timedelta(seconds=second)

    def all_on_lights(self):
        """
        Return list of entity_ids for all lights that are currently on
        """
        all_lights = self.get_state('light')
        all_on_lights = []
        
        for light in all_lights:
            state = self.get_state(light)
            if state == 'on':
                all_on_lights.append(light)

        return all_on_lights

    def get_max_brightness(self):
        """
        Return the maximum brightness value of all lights that are currently on
        """
        all_lights = self.get_state('light')
        max_brightness = []

        for light in all_lights:
            brightness = self.get_state(light, attribute='brightness')
            if brightness is not None:
                max_brightness.append(brightness)

        return max(max_brightness)

    def bound_to_255(self, number):
        """
        Convert percentage-bound rightness to something that is usable for
        HomeAssistant
        """
        return round(int(float(number)) * 255 / 100)

    def bound_to_100(self, number):
        """
        Convert HomeAssistant-usable brightness level to something that is
        human readable
        """
        return round(int(float(number)) / 255 * 100)

A lot of these are very simple, but allow you to understand how centralizing code in single app can help keep your other apps cleaner. There are various timers in here like soon() and tomorrow() that can help you standardize logic. There are also some functions in here that help you when working with light objects within HomeAssistant. These can be useful when you want to do automation based lighting.

In the app that wants to use your utility functions, you’ll want to get the app during the initalize section so we can access those functions elsewhere.

import appdaemon.appapi as appapi

class HelloWorld(appapi.AppDaemon):
    def initialize(self):
        self.utils = self.get_app('utils')
        self.say_tomorrow()

    def say_tomorrow():
        self.log('Tomorrow is {}'.format(self.utils.tomorrow())

Hopefully this give you the tools you need in order to start writing your own general purpose logic! If you’ve already got some of these type of functions, I encourage you to share with others in this thread!

#####*You can read up more about code re-use and docstrings conventions at their respective links below.
10 tips on writing reusable code - this article might go over your head a bit, but it’s useful information for all programming languages about writing good code that is intended to be re-used. Take the time to read it, and possibly come back to it as you find yourself getting more comfortable in Python.
__docstring__ - supplies classes for manipulating dates and times in both simple and complex ways.


— complex —

I’ll share with you a couple of functions I use (including some of the above in simple). There is a TON of flexbility when we’re talking about utilities and it really will depend on what you find you’re lacking from the general API.

import appdaemon.appapi as appapi
import json
from datetime import datetime, timedelta

#
# App to localize frequently used functions
#
# Args: (set these in appdaemon.cfg)
# n/a
#
# EXAMPLE appdaemon.cfg entry below...
#         ... and of how to make utils initialize before other apps 
#         get the chance to
# 
# # Apps
# 
# [utils]
# module = utils
# class = utils
#
# [hello_world]
# module = hello_world
# class = HelloWorld
# dependencies = utils
#
# class HelloWorld(appapi.AppDaemon):
#     def initialize(self):
#         self.utils = self.get_app('utils')
#         self.log('Tomorrow is {}'.format(self.utils.tomorrow())
#

class utils(appapi.AppDaemon):

    def initialize(self):
        # read into memory data from a configuration file
        # this data can be accessed in other apps via "self.get_app('utils').users"
        with open('/some/path/to/users.json', 'r') as j:
            self.users = json.load(j)

    def soon(self, seconds=None, minutes=None, hours=None):
        """
        Return datetime.datetime object for some time in the future from 
        current internal AppDaemon clock. 

        Keyword arguments:
        hours -- hours in the fturue (default None)
        minutes -- minutes in the future (default None)
        seconds -- seconds in the future (default 5)
        """
        if not seconds:
            seconds = 5
        if not minutes:
            minutes = 0
        if not hours:
            hours = 0

        return self.datetime() + timedelta(seconds=seconds, 
                                           minutes=minutes, 
                                           hours=hours)

    def run_every_weekday(self, callback, start, **kwargs):
        """
        Execute a callback at the same time every day of traditional work week. 
        If today is a work day and the time has already passed, the function 
        will not be invoked until the following work day at the specified time.

        Keyword arguments:
        callback -- Function to be invoked when the requested state change 
                    occurs. It must conform to the standard Scheduler Callback 
                    format documented at https://goo.gl/EBtPDx.
        
        start -- A Python time object that specifies when the callback will 
                 occur. If the time specified is in the past, the callback will 
                 occur the next day at the specified time.

        **kwargs -- Arbitary keyword parameters to be provided to the callback
                    function when it is invoked.
        """
        WEEKDAYS = ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday']
        handle = []
        upcoming_weekdays = []

        today = self.date()
        todays_event = datetime.combine(today, start)

        if todays_event > self.datetime():
            if today.strftime('%A') in WEEKDAYS:
                upcoming_weekdays.append(today)

        for day_number in range(1, 8):
            day = today + timedelta(days=day_number)
            if day.strftime('%A') in WEEKDAYS:
                if len(upcoming_weekdays) < 5:
                    upcoming_weekdays.append(day)

        for day in upcoming_weekdays:
            event = datetime.combine(day, start)
            handle.append(self.run_every(callback, event, 604800, **kwargs))

        return handle

    def run_every_weekend_day(self, callback, start, **kwargs):
        """
        Execute a callback at the same time every day outside of the 
        traditional work week. If today is a weekend day and the time has 
        already passed, the function  will not be invoked until the following 
        weekend day at the specified time.

        Keyword arguments:
        callback -- Function to be invoked when the requested state change 
                    occurs. It must conform to the standard Scheduler Callback 
                    format documented at https://goo.gl/EBtPDx.
        
        start -- A Python time object that specifies when the callback will 
                 occur. If the time specified is in the past, the callback will 
                 occur the next day at the specified time.

        **kwargs -- Arbitary keyword parameters to be provided to the callback
                    function when it is invoked.
        """
        WEEKEND_DAYS = ['Saturday', 'Sunday']
        handle = []
        upcoming_weekend_days = []

        today = self.date()
        todays_event = datetime.combine(today, start)

        if todays_event > self.datetime():
            if today.strftime('%A') in WEEKEND_DAYS:
               upcoming_weekend_days.append(today)

        for day_number in range(1, 8):
            day = today + timedelta(days=day_number)
            if day.strftime('%A') in WEEKEND_DAYS:
                if len(upcoming_weekend_days) < 2:
                    upcoming_weekend_days.append(day)

        for day in upcoming_weekend_days:
            event = datetime.combine(day, start)
            handle.append(self.run_every(callback, event, 604800, **kwargs))

        return handle

    def cancel_multiday_timer(self, handle):
        """
        Cancel a previously created weekday or weekend timer
        """
        for thing in handle:
            try:
                for timer in thing:
                    self.cancel_timer(timer)
            except TypeError:
                self.cancel_timer(thing)

Aside from a more flexible soon() timer, I’ve also included two timers I’ve found useful for running functions during weekdays and weekends. These are special timers that return a list of handlers, so you’ll need to use cancel_multiday_timer() in order to shut them off.

You’ll also notice that in my initalize section, I assign a variable to users. This variable can be accessed throughout other apps much like below.

import appdaemon.appapi as appapi

class HelloWorld(appapi.AppDaemon):
    def initialize(self):
        self.utils = self.get_app('utils')
        self.say_hello_to_users()

    def say_hello_to_users():
        for user in self.utils.users:
            self.notify_user('Hello {}!'.format(user)

If you’ve already got some of these type of functions, I encourage you to share with others in this thread!



Shout out to @aimc for developing this awesome platform we can use to write our complex automations in, and @yawor for helping to clean up and standardize the codebase. What we’re doing here would not be possible without these two, so thank you greatly.

I’ve gone back and forth on what medium to use, including various blogging platforms as well as YouTube. I might experiment with solving a problem “live” and thinking my way through the problem out loud as well.

Feedback is important! This series will only be as successful as you all make it to be! Let me know your thoughts and what you all would like to see, and I’ll consider all my options.

Happy Automating!

  • SN
13 Likes

Loving this - keep it coming :slight_smile:

Feels a bit like reinventing the wheel to me.
How about

Run_daily(callback,time,constrain_days=mon,tue,wed,thu,fri)
Or
Run_daily(callback,time,constrain_days=sat,sun)

But i still love that you write tutorials :wink:

2 Likes

You know – I haven’t really messed with the constraints. :smirk: I personally would prefer to write up my own function that go and learn how to format my contraints! It certainly can be useful though, to anyone who wants to see another way of doing it, @ReneTode is totally right. Check out constraints!

Admittedly, I was at a bit of a loss when trying to come up with useful utility functions for those of us that are more familiar with coding but thought it was a vitally important topic for the newbies. Utility functions are most useful to your specific setup. My person favorite utility is one that will send me a notification (via Join) to my phone if I am at home, but to my PC if I am work or on-the-go… however that is likely a small subset of people who have the same setup as me, so writing something that can be copy/pasted poses a bit more difficulty! :slight_smile:

Utilities can absolutely be usefull. And dont think i didnt reinvent the wheel several times :wink:
the most important thing i alwYs want people to know:
Only program that what you cant find on the internet
use a line 1 time, dont think about it, use it twice start thinking if you dont want to use it more.
if you think you want to use it more, then think if you can simplefy it.
it can be usefull to use 2 lines of coding in stead of 1, if it makes programming easier for you, but most of the times it is: less is more.

@SupahNoob - I’m not following what it is or where the users.json is coming from. Can you clarify? I’m sure I missed something.

Looks like it is in reference to your other post around custom Presence Listening, is that right?

It’s more as example of “read in a configuration file and store that data in memory, then access it from another app.” However my import json is in the wrong file. :slight_smile:

I’ve changed the file path to be a bit more ambiguous and given it a comment explaining the intent here.

Here are some utility functions that I like. Please note, Python is not my first language. I am sure there are better ways that make better use of “python” features.

  ######################
  #
  # build_entity_list (self, ingroup, inlist - optional: defaults to all entity types))
  #
  # build a list of all of the entities in a group or nested hierarchy of groups
  #
  # ingroup = Starting group to cascade through
  # inlist = a list of the entity types the list may contain.  Use this if you only want a list of lights and switches for example.
  #            this would then exclude any input_booleans, input_sliders, media_players, sensors, etc. - defaults to all entity types.
  #
  # returns a python list containing all the entities found that match the device types in inlist.
  ######################
  def build_entity_list(self,ingroup,inlist=['all']):
    retlist=[]
    types=[]
    typelist=[]

    # validate values passed in
    if not self.entity_exists(ingroup):
      self.log("entity {} does not exist in home assistant".format(ingroup))
      return None
    if isinstance(inlist,list):
      typelist=inlist
    else:
      self.log("inlist must be a list ['light','switch','media_player'] for example")
      return None

    # determine what types of HA entities to return
    if "all" in typelist:
      types=["all"]
    else:
      types= types + typelist
      types.append("group")            # include group so that it doesn't ignore child groups

    # check the device type to see if it is something we care about
    devtyp, devname = self.split_entity(ingroup)
    if (devtyp in types) or ("all" in types):                # do we have a valid HA entity type
      if devtyp=="group":                                    # entity is a group so iterate through it recursing back into this function.
        for entity in self.get_state(ingroup,attribute="all")["attributes"]["entity_id"]:
          newitem=self.build_entity_list(entity,typelist)    # recurse through each member of the child group we are in.
          if not newitem==None:                              # None means there was a problem with the value passed in, so don't include it in our output list
            retlist.extend(newitem)                          # all is good so concatenate our lists together
      else:
        retlist.append(ingroup)                                      # actual entity so return it as part of a list so it can be concatenated
    return retlist


  # delayed turn-on  - basically a wrapper for self.run_in with handler to turn on entity
  def turn_on_in(self,entity_id,delay):
    self.run_in(self.turn_on_handler,delay,entity_id=entity_id)

  def turn_on_handler(self,kwargs):
    self.turn_on(kwargs["entity_id"])

  # delayed turn_off - wrapper for self.run_in with handler to turn off entity
  def turn_off_in(self,entity_id,delay):
    self.run_in(self.turn_off_handler,delay,entity_id=entity_id)

  def turn_off_handler(self,kwargs):
    self.turn_off(kwargs["entity_id"])

  #####  Read jseon file named  filename  return dictionary
  def readjson(self,_filename):
    result={}
    if os.path.exists(_filename):
      fin=open(_filename,"rt")
      result=json.load(fin)
      fin.close()
    else:
      self.log("file {} does not exist".format(_filename))
    return result

  ##### Write dictionary out as a json file to filename
  def savejson(self,_filename,_out_dict):
    fout=open(_filename,"wt")
    json.dump(_out_dict,fout)
    fout.close()

  ##### Set unix file permissions
  def setfilemode(self,_in_file,_mode):
    if len(_mode)<9:
      self.log("mode must bein the format of 'rwxrwxrwx'")
    else:
      result=0
      for val in _mode: 
        if val in ("r","w","x"):
          result=(result << 1) | 1
        else:
          result=result << 1
      self.log("Setting file to mode {} binary {}".format(_mode,bin(result)))
      os.chmod(_in_file,result)
1 Like

Great tutorial.

I would like to see one on creating object classes.

class entity(appapi????)
  entity_name=""
  entity_state=""
  
  def _set_entity_name(self,name)
  def _get_entity_name(self)
  def _set_entity_state(self,name,state)  # or something like that
  def _get_entity_state(self, bla bla bla)

class switch(entity)
  def _set_entity_state(self,name,state) # overrides entity 
  def _get_entity_state(self, bla bla bla) # overrides entity, 

  def isgreater(self,entity,testvalue)   # returns boolean
  def isequal(self,entity,testvalue)     # returns boolean
  def isless(self,entity,testvalue)       # returns boolean

class light(entity)
    etc etc etc

Interesting. I don’t currently use class structure in my apps … in your example here it looks like you’re creating an entity/switch within an app, but I’m trying to understand how that’s more useful/effective than having this information all stored withing HASS itself. The only drawback of that approach is that it would persist through HASS restarts.

What are you trying to achieve with classes? In plain English, even if it’s long, I wanna hear it! :smiley:

ed//

These are great! Remember, documentation is important for anyone trying to read your code, so I’m going to change your comments from above the functions to actual docstrings. Love these ones! I might have to steal it. :slight_smile:

    def turn_on_in(self, entity_id, delay):
        """
        Call self.turn_on() for entity_id in a number of seconds.

        Wraps self.turn_on() in self.run_in(seconds=duration).
        See turn_on_handler() for more context.
        """
        self.run_in(self.turn_on_handler, delay, entity_id=entity_id)

    def turn_on_handler(self, kwargs):
        self.turn_on(kwargs["entity_id"])

    def turn_off_in(self, entity_id, delay):
        """
        Call self.turn_off() for entity_id in a number of seconds.

        Wraps self.turn_off() in self.run_in(seconds=duration).
        See turn_off_handler() for more context.
        """
        self.run_in(self.turn_off_handler, delay, entity_id=entity_id)

    def turn_off_handler(self, kwargs):
        self.turn_off(kwargs["entity_id"])

Let me help you clean these two up a bit. They get the point across, but are a bit prone to error. Additionally, filename isn’t a reserved keyword, so you don’t really need to have the leading underscore.

    def read_json(self, filename) -> dict:
        """
        Read json file into memory, returns dict.
        """
        if os.path.exists(filename):
            with open(filename, 'r') as j:
                return json.load(j)
        else:
            self.log("file {} does not exist".format(filename))

    def save_json(self, filename, out_dict):
        """
        Write a dict to file.
        """
        with open(filename, 'w') as j:
            json.dump(out_dict, j)

In most cases, filemode w and wt are the same thing, as the default in Python is wt or rt. :slight_smile: I do want to encourage you to use the with open() as syntax however, as it’s essentially the “foolproof way” to do things here. If an exception were to occur underneath your with block, Python will guarantee we call j.close() and that is certainly not the case in your example.

We haven’t, and likely won’t, talk about function annotations but it can be a decent help for those who aren’t familiar with your work and aren’t exactly sure what to expect what the function returns.

It’s really nothing that I can’t do with utility functions I guess. In a way that’s what I’m trying to do I guess. Classes give us a way to encapsulate the differences between objects and simplify our code. A Class is nothing more than a group of utility functions specific to an device-type in our case.

Let me start by saying I really like AppDaemon and the dashboard. They are great tools and provide a level of flexibility to HA that isn’t available on any other home automation platform. I could not do half of the things I am doing without AD. I have nothing but respect and admiration for the people on this forum and enjoy our discussions. I have learned a lot and hope to keep learning more.

I’ve re-written the next part of this three times now. I’m just gonna stop here, before I get evicted. LOL

3 Likes

No evictions :slight_smile: I am really happy to listen to anyone who cares enough to discuss/argue about what we are doing here - we all learn more that way. I measure any discussion, not by whether or not I “win” but whether or not I learned something - I never want to be “the smartest guy in the room”, because they learn nothing …

3 Likes

I understand, sometimes I just feel like I’m using C++ to write straight C code.

Also, please realize that for me, I’m about to be hitting the job market. So I’m using my interest in home automation to fuel my learning of a new language. To help make me more marketable. Also, home automation development is an interesting hobby to put down on the resume that may spark some conversation.

Sometimes my how would you do this questions are really how would you do this and while it may not be the best way to do things in AD, it teaches me more about the language and who knows may point out a new way of doing something we hadn’t thought about.

2 Likes

My advice would be to put “IoT” on your resume, very marketable term.

And as I said, always happy to have the discussion :slight_smile: