App #8: Detect a particular sequence of events

there are many moments that you want to act when 3 or more things happen in a row.
but the most obvious are movements.
motion livingroom > motion hallway > motion bedroom > someone moved to bedroom action
against
no motion livingroom and hallway in the last few minutes > someone moves in bedroom action.

@Burningstone allthough your code is probably very correct, its quite hard to read for noobs.
there is 1 thing i see:
a run_in (or any other schedular) expects

def name(self, kwargs):

and not

def name(self): 
1 Like

I’m working on making my code more readable for noobs, the problem is I’m a noob myself haha
Any tips or suggestions regarding this?

Thanks for pointing out the mistake, I quickly added the delay after the app was already finished and didn’t test it afterwards, that’s why I probably didn’t notice.

i am also noob :wink:

what i always try to do:

  1. explain everything you do and why you do it that way.
  2. if code isnt needed for the app then leave it.
  3. dont use oneliners (they are never self explaining)
  4. avoid complicated structures if possible
  5. if you can split things up to smaller blocks they do so

for example this part:

APP_SCHEMA = vol.Schema({
    vol.Required(CONF_MODULE): str,
    vol.Required(CONF_CLASS): str,
    vol.Required(CONF_TIMEFRAME): int,
    vol.Required(CONF_ACTION): vol.Schema({
        vol.Required(CONF_ENTITY_ID): str,
        vol.Required(CONF_SERVICE): str,
        vol.Optional(CONF_DELAY): int,
        vol.Optional(CONF_PARAMETERS): dict,
    }),
    vol.Required(CONF_EVENTS): vol.Schema({
        vol.Optional(str): vol.Schema({
            vol.Required(CONF_ENTITY_ID): str,
            vol.Optional(CONF_TARGET_STATE): str,
            vol.Optional(CONF_RANK): int,
        }, extra=vol.ALLOW_EXTRA),
    }),
}, extra=vol.ALLOW_EXTRA)

is really hard to see what it is for, why you use it and what it does exactly.
and also a bit overkill for a simple app like this.
if you use something like voluptuous in such an app, then take the time to give a simple description from what it is, and what it does with a link to the voluptuous docs.

real noobs wont understand why you use:

    APP_SCHEMA = APP_SCHEMA

more logical to read is if you put that in the initialise like:

    self.APP_SCHEMA = APP_SCHEMA

in general i also try to avoid EVERYTHING that is done outside the class (app code) except imports off course.

why create extra lines?

CONF_ACTION = 'action'

and then later on use the constant. it makes code less readable, because i need to read back every time i see a constant, to know what it stands for. constants like that are nice if you want to use them for translatable output, but not for simple apps like these.

avoid the use from names that are used in a different setting.
a noob knows that attributes can be a part from an entity. now you use it as a part from an event.
better readably is if you call them event_settings.

explain when you add kwars to a listener and when you use them. a noob wont understand how you get to

        rank = kwargs[CONF_RANK]

this is probably very correct:

    def update_event_trigger_timestamp(self, rank: int) -> None:

but if you look at forums or python scripts you find elsewhere you would probably find:

    def update_event_trigger_timestamp(self, rank):

the extra characters are nice if you write for yourself or if you share code, but not for showing options.

in a lot of cases a few extra line will make things more self explaining

     def all_events_triggered_in_timeframe(self) -> bool:
        """Returns true if all events triggered within timeframe and in order."""
        return (
            self.all_events_triggered and
            self.in_timeframe and
            self.triggered_in_order
        )

this works perfectly, but it would be more readable if you use:

    def all_events_triggered_in_timeframe(self) -> bool:
        """Returns true if all events triggered within timeframe and in order."""
        if self.all_events_triggered and self.in_timeframe and self.triggered_in_order:
            return True
        else:
            return False

i hope this helps you understanding why i said that your code isnt easy to read for noobs.

your code is far away from noob code. you did take a look at advanced coding (and looking at the style, HA code) and use those structures and usage. nothing wrong with that, but its far away from noob code. (nothing wrong with that, unless your goal is to learn others using apps :wink: )

I struggle with this myself.

Like @Burningstone I use voluptous in most of my AppDaemon apps. It’s not easier or more readable than not using it, but it does all the heavy lifting of making sure that floats are floats when a float is expected, or that strings placed where a list should be are converted to a single item list, etc, etc. And it does a good job of sending a useful error message to the user if they accidentally type a configuration item in their YAML incorrectly. It also makes it so that, later in my code, I don’t have to validate any inputs because I typed:

timeout_seconds: "50"

instead of…

timeout_seconds: 50

So, no, it isn’t noob friendly to read. But it’s noob friendly to use. To make it more readable… I like to do this…

class someThing(hass.Hass):
  def initialize(self):
    self._config = self.validate_configuration()

  def validate_configuration(self):
    APP_SCHEMA = vol.Schema({
      vol.Required(CONF_MODULE): str,
      vol.Required(CONF_CLASS): str,
      ... blah ...
    })

    return APP_SCHEMA(self.args)

At least, this way, if a noob can’t understand exactly what it’s doing, at least they understand the gist of it because of the method name.

For,

CONF_ACTION = 'action'

I’ve always hated this. I get it. It makes it so a configuration element’s name can be easily changed later, assists in allowing multi-language support, and can sometimes make it clearer when I say self._config[CONF_ACTION] that I’m using the action passed in from configuration. But, personally, it still bugs me to read. If I’m using someone else’s code as a blueprint and it’s written that way, then I keep doing it. But if I’m writing from scratch, I don’t.

i wasnt even familiar with it, but its like a lot of other programs that are out there making programming more easy, it makes people sloppy.
noobs in particular shouldnt use those things. learn to program without help from those things and you will be more carefull.
you yourself wil make sure that a float is on the place from a float or convert when needed.
if we dont validate our input ourselves very carefully, we are on a sliding path.

putting the validation in a seperate function at least makes it more readable, thats right.
i can see the use in big programs with lots of input, but i dont think its very helpfull in small apps.
i rather see a description on the top of the app like:

#####
# this app does something.
# to make this app work it need the following yaml:
#  action: (the action that will be done)        - required
#    entity_id: some.entity                      - required
#    service: 'turn_on'                          - required
#    delay: 60                                   - optional, will default to 60, NEEDS to be an INT
#    parameters:                                 - optional, parameters need to be usable with the service
#      brightness: 10                            - optional

another big point is that if you show that you are doing the validation for the user, he will expect that you do it completely and blame you when it isnt working.
for instance in the code from burningstone you can give parameters for the service.
it expects a dict. so anything will work. as long as it is a dict.
so as a user i expect that

    parameters:     
      brightness: "10"

would work or at least give me an explaining error. but it doesnt.
thats a problem.
so to use voloptuous correctly, the possible parameters need to be ALL specified.
but then the code for a small app would be endless.

Thanks a lot for your suggestions. Highly appreciated!

It seems like I need to rethink, what the goal of my code should be. My tendency goes in the direction of noobs being able to use my apps, but not necessarily need to understand the code behind the app.

Documentation of my code is still one of my weakest points, but I’m working on it :sweat_smile:
I should probably add in the beginning of the code a description of the general purpose of the app and how the config needs to look like.

Regarding voluptuos, I have a core app from which all my apps derive configuration and other things and in which the config validation is done as well, so I wrote the app for my setup and then just adjusted it, so that it would also work for others. I should have excluded this from the code I posted, because as you said it is confusing for people not familiar with it.
I use voluptuous for all my apps no matter how small they are, I just got so used to it and was tired of all these:

if 'entity_id' in self.args: 
    do something
else:
    self.log("Entity id is missing")

You made a good point here about the validation not catching all the possible wrong configurations.

I’m still thinking of a way to validate this as well, but I have no clue how I could achieve this due to the almost endless possibilities of parameters one can give for service calls. You have any suggestions for this?

Thanks for your suggestions. Highly appreciated!

I saw this a lot in other peoples code and then I started doing it for some of my own apps and now I’m so used to it that I do it automatically, but I see now the confusion and complications this creates. I will try to reduce the use of this, but I have a feeling that it will be hard to get rid of this habit :sweat_smile:

Voluptuous IS input validation. By using it, I am validating the input myself. It’s just a nice pre-written package that makes it easy to do so.

Right. Documentation is important regardless of what tools and libraries you’ve used when programming. But, somewhere in your code you have to check to make sure the required variables have been provided, and that they are of the correct type. Voluptous doesn’t mean we should leave the example YAML out when documenting an App, it just means I use Voluptous to ensure they’ve been included instead of a bunch of ifs and self.log("blah", level="ERROR") code everywhere.

If you use Voluptous correctly, it DOES work. And if for some reason it doesn’t work (perhaps because it wasn’t used correctly, or because the code author specifically doesn’t want to allow this), it DOES give you an error explaining. And the code for a small app isn’t endless. That’s the whole point.

Here’s an example usage of Voluptous in one of my apps:

        schema = vol.Schema(vol.All(
            appdaemon_schema.extend({
                vol.Required('rooms'): {
                    vol.Any(str): {
                        vol.Optional('occupied', default=None):
                            vol.Maybe(str),
                        vol.Required('temp'): str,
                        vol.Optional('multi', default=1): int,
                    }
                },
                vol.Optional('sample_interval', default=60): int,
                vol.Optional('max_intervals', default=10): int,
            })
        ))

        self._config = schema(self.args)

If you don’t use or understand Voluptous, this might make no sense at all. However, if I also provide sample YAML, it helps:

app_name:
  module: min_max_temps
  class: MinMaxTemps
  rooms:
    living:                                    # required - name of the room
      occupied: binary_sensor.living_occupied  # optional - entity that indicates a room is occupied
      temp: sensor.living_temperature          # required - entity providing the temperature of the room
      multi: 5                                 # optional - default 1 - provides additional importance to the temperature in that room 
    kitchen:
      occupied: entity_id
      temp: entity_id
  sample_interval: 60    # optional - default 60 - how often temperatures should be checked
  max_intervals: 10      # optional - default 10 - number of samples to average together

With these two things, the user knows how to use my App AND all of the user inputs are validated and any errors are reported. If you spell “room” as “rooom” on accident, you’ll see an error. If you write multi: five instead of multi: 5, you’ll see an error. If you write multi: "5" instead, you’ll also see an error because I didn’t tell Voluptous to convert it to an Integer. But I could have. I should have. And with this one change to my code, it’s now allowable:

vol.Optional('multi', default=1): vol.Coerce(int)

I’m not saying everyone HAS to use Voluptuous. There are a lots of ways to accomplish a goal. But it certainly makes it easier for me. And, while, yes, it makes the code a bit less readable for those that don’t know Voluptuous, documentation is always needed as well, regardless of what libraries are used.

If you use Voluptous, Coerce() will do most of the type correction for you. This code requires an int:

vol.Required('seconds_until_shutdown`, default=0): int

However this code, will attempt to convert whatever type was provided to an int, and only error if it can’t do so.

vol.Required('seconds_until_shutdown`, default=0): vol.Coerce(int)

It works the other way too. vol.Coerce(str) will turn 5 into "5" and 7.123 into "7.123". And, for instance, vol.Coerce(int) will turn "5" into 5 but it will not turn "five" or "hot dog" into a usable number. Instead, it will provide an error.

I think it’s good to have both options out there. I’ve created a few generic apps that are available in HACS, so that noobs can use the power of AppDaemon without having to touch any Python. I think that’s a huge benefit of AppDaemon, so having apps out there that are usable by anyone without understanding of the code is definitely a good thing. At the same time, for those who want to learn and expand their knowledge to create their own automations in AppDaemon, it’s great to have more documented examples. I haven’t done much on that side yet, but it’s definitely on my to-do list to take some deeper dives into stepping through apps I’ve written for those who want to learn.

1 Like

I think a lot of people do it because Home Assistant does it. In fact, if you try to submit a PR that doesn’t do it, they’ll request that you change it. Their reasoning is that CONF_NAME for instance, is not defined in your code. It comes from the core of Home Assistant. So if the core changes CONF_NAME = 'name' to CONF_NAME = 'name_of_the_thing' it would change in your code as well.

It’s biggest use case is in language. English can have CONF_NAME = 'name' while Spanish has CONF_NAME = 'nombre'. Then, in your code, you just use CONF_NAME and your code will automatically adjust to allow configuration in whatever language is being used. However, Home Assistant doesn’t actually use this. And, even if they did, if you used a configuration element that wasn’t already included in home assistant core, then you’d either have to implement every language yourself or someone using Spanish would still have to use the English name for that parameter which would be more confusing, I’d think.

And, if Home Assistant did decide to change CONF_NAME = 'name' to CONF_NAME = 'name_of_the_thing' more than likely they’d just add CONF_NAME_OF_THE_THING = 'name_of_the_thing' and then change all the code that uses CONF_NAME to use CONF_NAME_OF_THE_THING.

So, it’s a valid idea, with valid uses, but, None of those valid uses are being employed, so it just makes it confusing.

thats exactly my point.
its very hard to get the validation complete in such cases. i wouldnt even start thinking about it.
and bad or incomplete validation is in my eyes worse then no validation.

no by using it you try to avoid the need to do it yourself. you let the program do your work, but that only works if the program is complete and correct.

yup but there are cases where its almost impossible to do it correct.
like in the code from burningstone.

APP_SCHEMA = vol.Schema({
    vol.Required(CONF_MODULE): str,
    vol.Required(CONF_CLASS): str,
    vol.Required(CONF_TIMEFRAME): int,
    vol.Required(CONF_ACTION): vol.Schema({
        vol.Required(CONF_ENTITY_ID): str,
        vol.Required(CONF_SERVICE): str,
        vol.Optional(CONF_DELAY): int,
        vol.Optional(CONF_PARAMETERS): dict,
    }),

it checks if parameters is a dict. the parameters are used for a service.
if i use:

parameters:
  bright_light: 2598

volop wil accept it without any error.
its however impossible to specify ALL possible parameters from all possible services.
but you have validation so the user expect that he cant go wrong in his config without errors.
so any error that comes up or any situation that isnt working, the user will blame the code and not his input.

and i think you shouldnt. its like the autocorrect function from a smartphone. the result is that people dont look themselves anymore and send texts that they didnt want to send.

i am not saying people shouldnt use it in general. but if you present code to help people learn coding, then i say: let it out.

thats my point exactly. it can be usefull, but if you dont use it then its just obsolete code.

This is like saying “By using AppDaemon, you avoid doing the work yourself”. Or “by using indoor plumbing you avoid doing the work yourself”. It’s a tool. Tools are meant to make the work easier. And voluptuous does exactly that.

Voluptuous accepts ANY dict, because that’s all that was requested. If the Schema is designed this way, then either any dict is truly valid, the programmer didn’t do his/her job, or the CONF_PARAMETERS field is validated another way.

Yes and no. If your goal with the app is to allow any service call at all, without knowing the available parameters ahead of time, then with or without Voluptous there are “parameters” that will break when running. If you wanted to do this “right” you’d have config (or an API endpoint you could use to verify) that contained all available services and their parameters. That point is true regardless of if you’re using Voluptous or not. Or take the “lazy” route (which is perfectly acceptable in many cases) and just make the service call and handle the error if there is one. Using a tool like Voluptous doesn’t remove your need to validate input, it just changes the way that validation looks in code.

And that’s fine. If you think that "5" shouldn’t be accepted when 5 is what is needed, you can write your app that way. Voluptuous is input validation. It’s up to the programmer to decide what’s valid and what isn’t. You could do the same thing without it… accept the string, or accept only an int. The choice is the programmers. But, either way, the code should give an error message if the input is not selected. Voluptous does this for you. If you don’t use Voluptuous, you should do that yourself.

in some cases. but in a lot of cases it doesnt.
if i write an app that i only use myself, maybe even 2 times, then its more work to add the voluptous code then to check my parameters.
for most apps i come across i think voluptous is overkill.
i think in lots of cases the use of voloptous is more like: creating an appdaemon app to switch on a switch that you only switch on once. or like building a robot to repair the hole you have drilled in your plumbing.

and thats the problem there is none, and you let people think that there is correct validation, so in a case like that you have to do an enourmous amount of work to get all possibilities and check for them, or you tell people that they need to make sure that their input is correct. because you cant validate anything at that point.

of course the code will give an error if a user uses a string, where an int is expected.
do i need to catch that error and explain in the log where the user did go wrong?
hmm, yeah if i get paid for the program i need to do that. but thats not the case, so i just tell in the description that an int is needed. when the user uses a string in that case, he will get an error that he might or might not understand. if he comes to me with it, i understand it and i will say: i told you to use an int, dummy.

a matter of choice :wink:

but i think we have highjacked @rhumbertgz topic enough already and we all made our points.
i think we can agree that there is a big difference between teaching people to write simple apps, and write apps that people can use.

I agree with this mostly. If I’m writing a quick app to do one quick thing, I often don’t even bother with parameters in YAML. I just hard code it all and done. It’s just me. It just does one thing. And there’s nothing to validate since there are no parameters.

If I need to use the same app again, I could just copy the code and change all the hardcoded things, but I don’t. I generally make it more generic at that point. I’ll add parameters and such so that I can use it multiple times. And now, because there are parameters, I have to deal with input validation. If it’s quick and something only I will use, I might not bother with input validation at all. I do, however, bother to copy self.args to self._config (my standard place to store validated config) so that, in the future, if I do add input validation, I don’t have to change all the code below to reference the validated dict.

But, if I get into a situation where either I think others might be able to use the app, or I need a parameter to work in different ways in different use cases, then I add input validation, and I almost always use Voluptuous to do so. It’s actually easier for me than doing it by hand now that I understand it.

I don’t let them think there is proper validation any more than I would if I didn’t use Voluptuous. It’s exactly the same in that regard.

100% agree. I’m happy to teach Voluptous. And I learned to use it by looking at the code of others. But, if it’s literally your first AppDaemon App, it’s overkill.

1 Like