HA automation in Python from a developer's POV

[molto_b (Mike Pagel)]

The challenge

HA allows various ways of scripting and automation. I am still blown away of how much stuff can be done in the HA application itself, be it the UI, an embedded editor for YAMLs or extensions like NodeRed.

If you’re proficient with Jinja templating and YAML you’ll get things done. I am. But is that a development experience I am completely happy with?

Not quite, as I keep comparing this to a native development environment. So questions like this might come up:

• I can discover many options in the UI, but eventually, I’ll end up using YAML configs. How do I easily discover APIs and constructs without code completion (or browsing docs all the time)?
• How can I cleanly structure reuse? Yes, we can create and call (utility) scripts, services and automations, but I believe all these items live at the same (top) level. How do I ensure I’ll still find stuff? And blueprints are not more than gists I clone. That’s hard to maintain.
• More complex logic might deserve some (automated) test code beyond live rollout and running through the house to check.
• Other quality tools (black, linters, …) might be interesting.
• How can I debug? Can I set a breakpoint to inspect the data that is passed to my automation? The tracing feature for automations is terrific but it’s not a debugger.
• Maybe I want to use my own IDE?

As a Python developer I looked at the options to script automations in Python, but the above kept nagging me. I wanted a real developer experience.

Python automation options

I looked at these:

• The built-in Python scripts integration is a good first start, but quite limited in features.
• PyScript is much more capable and a very cool project. It’s only a subset of Python though and one of the things it lacks is support for classes (hope I did read this correctly). I’d like to use classes for organization of my logic and not wholy rely on function composition.
• AppDaemon is real Python and does have reuse in mind. As an externally running application it is limited in terms of what it can do within HA, but on the other hand, it is decoupled. And that allows us to run it from anywhere… IDE, I’m coming!

There were more options that crossed my path, but none of them seemed to be in use very much, so I’ll leave them out.

My setup

AppDaemon (AD) is a stand-alone application written in Python. It talks to HA over a web (socket) API. This allows running an instance of AD on your local development machine. In fact, you can run one AD within your production setup and connect with another development instance to the same HA, if you like.

The setup of the development machine is:

1. Create a virtual environment and install the appdaemon package. Immediately, you are able to call appdaemon -h to get the CLI help page.
2. Install your preferred development tools into that venv.
3. AD is looking for configuration to load, typically from a folder called config, create that as part of your working area. It usually contains this content:
   • The appdaemon.yaml file. It will be almost identical to the one on the production system, just the own http interface should be set to http://127.0.0.1:5050.
   • An apps folder. This is where your AD apps will be created.
4. I use git not only for versioning, but also to (literally) pull updates onto the production system. Watch out to not overwrite anything but AD apps. I use a Github repo as shared repository.
5. In the IDE/editor or your choice, consider creating run/debug configurations.
6. Take a look at the -D option of calling AD, it allows you see your module’s debug log messages in the console.

Having this in place, start reading the AD docs. For me the best initial page was Writing AppDaemon Apps. It explains most of the concepts to get a good first grasp.

And now, start coding, set breakpoints, go to definitions of AD methods to learn, how they do things etc.

To get the best experience (at least in PyCharm), I am using the asynchronous approach (all callbacks are declared as async def). Besides providing good code completion this allows for very simple sequence programming, as asynchronous sleep can be used.

Here are some impressions of the development experience:

---

Prepared run configuration

---

Logs including debug output of automation code

---

Inspecting a breakpoint in a callback

---

Code completion

Wrapping up

I hope this can be useful for some developers. It would have helped me getting over that initial difficulty of choosing a Python development solution. Let me know how you do development of automations and of course help me correcting false statements or assumptions I might have made.

[dubhunter (Will Mason)]

@molto_b you are a genius. This is a total game changer.

I finally (after too long) decided to sit down and get AppDaemon working tonight. While working on my first “real” app, I immediately wanted my debugger. A few failed google searches later, I found your post. A little tinkering and I just hit my first break point.

Thanks for your post!!

[zzafarr (ZB)]

@molto_b How do you debug with AppDaemon?

Also, is there a way (easy ) to write the Unit Tests ?

Thanks!

[molto_b (Mike Pagel)]

I’m sorry I missed your question until now!

Yes, totally, you can write tests. I personally do not test (hence mock) the interaction with AD. I am mainly interested in validating internal logic, i.e. my own code. Here I also have control over whether async is used or not, possibly making my life easier as well.

Hope this gives some insight.

[brazoayeye]

thanks for informations!

I installed the HASS add on, copied the appdaemon.yaml from there but it’s very small and missing all connection informations

---
appdaemon:
  latitude: 42.779189
  longitude: 6.599431
  elevation: 2
  time_zone: Europe/Amsterdam
  plugins:
    HASS:
      type: hass
http:
  url: http://127.0.0.1:5050
admin:
api:
hadashboard:

If I stop the plugin and I try
venv/bin/python3.11 -m appdaemon -D DEBUG -c .
i can’t connect because I haven’t specified connection informations.

Do you know how I can extract the informations from the plugin?
On the other side do you have a guide to understand needed parameters?

After that, can you send me the launch.json for vscode or a link explaining how to setup it for the debug?

Thanks

[molto_b (Mike Pagel)]

Under appdaemon I you’re missing where it should connect. The http section is for appdeamon’s HTTP interface, but appdaemon wants to talk to HA and you need to tell it where.

Here is my appdaemon YAML, I replaced values with angular bracketed expressions, where you need to add your data:

appdaemon:
  # this is important to get right to let HA/AD work with sunrise/sunset:
  latitude: <...>
  longitude: <...>
  elevation: <...>
  time_zone: <...>
  plugins:
    HASS:
      type: hass
      # you might need to ignore cert warnings, or switch the URL to http:
      # cert_verify: false
      ha_url: https://<url or IP where HA listens>
      token: <long lived token created in HA>
  <more config irrelevant here>
  # this might help if you find the kwargs handling in AD weird, see AD docs:
  use_dictionary_unpacking: true

<...>

And this would be the entry in my launch.json:

        {
            "name": "appdaemon",
            "type": "python",
            "request": "launch",
            "module": "appdaemon",
            "args": [
                "--config",
                "config",
                "-m",
                "radiator_guestbath",
                "DEBUG",
            ],
            "cwd": "${workspaceFolder}/../..",
            "console": "integratedTerminal",
            "justMyCode": false,
        },

And finally, this is the folder structure (files partly omitted) I am working in and referenced by the launch config: