Announcement: AppDaemon 4.0.0b1

Third party integrations AppDaemon
1

Hello Everyone -

It’s been a while since we have had a release but we’ve been working hard behind the scenes, and are now ready to beta the next major release of AppDaemon. As usual for a major release, there will be some breaking changes, so check the release notes and the docs. We have been using this version for quite a while without issues and think we have squashed most bugs, but as always, this is BETA code, so take care, and let us know if you run into any issues.

There are too many things to talk about here, but some of the highlights are:

Updated docs for AD 4.0 can be found here.

Installation instructions can be found here - PyPi has a development build available, use appdaemon==4.0.0b1

Some hints and tips on upgrading can be found here.

Beta 2 Spoilers

We are still hard at work and have some good stuff for the next beta:

  • A new camera widget that should behave better and work with a wider range of cameras
  • An input_text widget
  • An input_datetime widget
  • Initial support for ASYNC calls in apps

As ever, thanks to the regulars on the AppDaemon dev team:

  • Rene Tode
  • Odianosen Ejale

And introducing our 2 newest dev team members:

  • Humberto Rodríguez Avila
  • Daniel Lashua

Here is the full list of changes:

4.0.0 Beta1 08/30/2019

Features

  • Apps can now use a simplified version of the import statement e.g. import hassapi as hass or import mqttapi as mqtt. The existing import method will continue to work.
  • Apps can now use multiple plugin APIs with the get_plugin_api() function
  • Added ADBase superclass for apps that want to use the get_plugin_api() style of coding
  • Scheduler rewritten to be more efficiant and allow for microsecond resolution
  • listen_log() now sends AppDaemon system messages and has the option to set a log level.
  • Bumped aiohttp to v3.4.4
  • Added callback locking decorators
  • Rearchitected the work Q to allow App pinning and avoid re-entrant and concurrent code if desired
  • Implemented multiple worker Ques to avoid Head of Line blocking
  • API Calls to control app pinning
  • Added the run_in_thread() api call - with assistance from Odianosen Ejale <https://github.com/Odianosen25>__
  • reworked log listening functions to be more robust and added the ability to have multiple callbacks per app
  • Refactored plugin APIs to remove duplication
  • Moved contrain_days from being Hass only to all app, regardless of plugin used
  • Added checking for overdue threads
  • Added error checking for callback signatures
  • Added app attributes that allows to access AD’s config and apps directories within apps
  • Added parse_datetime()
  • run_once(), run_at() and run_daily() now optionally take parse_time() or parse_datetime() style arguments for specifying time
  • Refactored appdaemon.py for greater readability and easier maintenance
  • Expanded on the ability to trigger listen_state callbacks immediately using the immediate flag, without need of specifing the new nor duration parameter.
  • Allowed to make use of attribute when using the immediate flag in listen_state
  • Added initial version of the Admin Interface
  • Added User Defined Namespaces
  • Rewrote logging to include user defined logs and formats
  • Added a unified http component to handle API, ADMIN and DASBOARD access on a single port
  • Added startup conditions to the HASS plugin
  • Added duplicate filtering for logs
  • Added standalone pidfile functionality
  • Added the ability to delete an AD app generated entity from any namespace
  • Added the ability to get the history of entities from HASS database
  • Added the ability to force a start of the MQTT plugin, even if not connected to broker at startup
  • Added the ability to set AD’s production_mode from within apps
  • Added the ability to start, stop, restart and reload apps from either other apps or REST API
  • Added the ability to register app services
  • Added sensors for different internal state of AD, that can be read by apps
  • Added Person widget
  • Much reworking of docs
  • Added register_dependency() for dynamic dependencies in apps
  • Added MQTT support for setting TLS version - contributed by Miguel <https://github.com/mdps>__
  • Added support for socketio for older tablet devices - inspired by algirdasc <https://github.com/algirdasc>__ and zarya <https://github.com/zarya>__
  • Added support for default and copy parameters in get_state() api call - contributed by Robert Schindler <https://github.com/efficiosoft>__
  • added a switch to disable the encoding of every log message to ascii - contributed by Ben Lebherz <https://github.com/benleb>__
  • Various YAML fixes and refactoring - contributed by Rolf Schäuble <https://github.com/rschaeuble>__
  • Allow more natural addition of commandline arguments to Docker and allow spaces - contributed by Christoph Roeder <https://github.com/brightdroid>__
  • Allowed for subscribing to MQTT events using wildcards. e.g. homeassistant/# - contributed by Odianosen Ejale <https://github.com/Odianosen25>__
  • Allow to specify a MQTT message to be sent when AD shutsdown cleanly e.g. offline
  • MQTT Retain setting for birth and will messages - contributed by Clifford W. Hansen <https://github.com/cliffordwhansen>__
  • Added Note on long lived tokens for Docker users - contributed by Bob Anderson <https://github.com/rwa>__
  • Documentation fixes - contributed by Johann Schmitz <https://github.com/ercpe>__
  • Documentation fixes - contributed by Brendon Baumgartner <https://github.com/bbrendon>__
  • Documentation fixes - contributed by Quentin Favrie <https://github.com/tseho>__
  • Documentation fixes, updating and cleaning - contributed by Humberto Rodríguez A. <https://github.com/rhumbertgz>__
  • Added the ability to set title 2 as friendly name in widgets - contributed by Radim <https://github.com/rds76>__
  • Added the ability to listen to state_change events, without using listen_state() - contributed by Thomas Delaet <https://github.com/thomasdelaet>__
  • APIAI updated to dialog flow - contributed by engrbm87 <https://github.com/engrbm87>__

Fixes

  • Fixes to listen_state() oneshot function
  • Fixes to listen_state() oneshot function when duration is used
  • Fixes to listen_state() function when it fires even when new and old states are same
  • Fixed an issue causing incorrect busy thread counts when app callbacks had exceptions
  • Fixed an issue of when MQTT Plugin not connected to broker, and it holds up AD startup
  • Fix to Forcast min/max in weather widget - contributed by adipose <https://github.com/adipose>__
  • Fix climate widget docs - contributed by Rene Tode <https://github.com/ReneTode>__
  • Fix to harmonize units vs unit - contributed by Rene Tode <https://github.com/ReneTode>__
  • Added missing import in sound.py example - contributed by cclaus <https://github.com/cclauss>__
  • Fix for run_once() - contributed by engrbm87 <https://github.com/engrbm87>__
  • Fix for onclick not working on IE11 - contributed by jgrieger1 <https://github.com/jgrieger1>__
  • Fixed issue of AppDaemon loading all .yaml files, even those starting with a . which are hidden or binary files. Contributed by fhirschmann <https://github.com/fhirschmann>__
  • Fix for error generated when a none existent schedule timer is passed to info_timer
  • Fix for log_type flag in listen_log callback
  • Relative paths for appdaemon’s config directory now work corrcetly
  • Fix to Dialogflow after format changes
  • MQTT fix to subscribing using wildcards - contributed by Daniel Lashua <https://github.com/dlashua>__

Breaking Changes

  • appapi.py has been renamed to adbase.py, and the contained superclass ha been renamed from AppDaemon to ADBase. This should only be a breaking change if you were using unpublished interfaces!
  • Time travel semantics have changed to support faster scheduling.
  • plugin_started and plugin_stopped now go to the appropriate namespace for the plugin and are no longer global
  • Apps are no longer concurrent or re-entrant by default. This is most likely a good thing.
  • Changed the signature of listen_log() callbacks
  • cancel_listen_log() now requires a handle supplied by the initial listen_log()
  • Removed Daemonize support - please use sysctl instead
  • set_app_state() is deprecated - use set_state() instead and it should do the right thing
  • dash_compile_on_start now defaults to true
  • The log section of appdaemon.yaml has been deprecated and must be replaced by the new logs section which has a different format to allow for user defined logs and greater flexibility in formatting etc.
  • API no longer has a separate port, all access is configured via the new unified http component
  • API has its own top level configuration section
  • Some dashboard parameters moved to the HTTP section and renamed
  • dash_compile_on_start renamed to compile_on_start
  • dash_force_compile renamed to force_compile
  • Due to the new log parameter to allow apps to use user defined logs, any previous parameters named log should be renamed
  • Due to a fix for info_timer, this function can now return None if the timer handle is invalid
  • As a result of a change in the way AD auto generates MQTT client status topic, if not defined previously the new topic needs to be used
  • In the appdaemon configuration section, latitude, longitude, elevation and timezone are now mandatory
  • MQTT client status api change from clientConnected to is_client_connected
16 Likes
2

Very smooth cutover from 3.0.5 to 4.0.0b1. Excited to test further (and for what’s to come – asyncio!).

2 Likes
3

Thanks for the feedback Aaron - you are going to love what we have planned for Async apps :slight_smile:

2 Likes
4

I’m trying to make sure we cover as many use cases as possible with our asyncio implementation. So, if you’ve got a moment please tell me, how would you like to use asyncio in an AppDaemon app?

5

Appreciate the reach-out, Daniel!

Off the top of my head, I’d like to be able to:

  • Define async listener callbacks
  • Define async scheduler callbacks
  • Access the primary event loop that AD uses (to add tasks, executor jobs for sync functions, etc.)
6

Evening, How do i find out what version of AD i’m running

7

start appdaemon, and look at the logs.
if you are on hassio, then dont get confused by the addon version, which gives loglines before AD is starting (and if you are on hassio you will probably be on 3.0.5)

8

Thanks. just found it.
I’m running AD in a docker on unraid from the acockburn/appdaemon repository, can i still try the beta?

9

Fantastic! We got those three cases covered in “dev” right now and they will hopefully be making it to beta soon. A few highlights:

  • both “initialize” and “terminate” can be async methods.
  • all callbacks can be async methods.
  • and calls made to methods like self.call_service(), self.get_state(), etc return futures and can be awaited when called from within an async method
  • all async methods access the primary event loop
  • from sync methods you can add an async task using self.run_coroutine(). It returns a future and supports calling a callback when complete for flexibility.

The only thing on this list that isn’t supported is adding an executor job to the event loop. From within a sync method, you can just call the sync function and it’ll be handled in the AD thread for that app. From within an async method you can use asyncio.run_in_executor() but we haven’t yet made this “easy”. I’ll look into the possibility, though.

Thank you for your feedback!

1 Like
10

I am sorry but i am no docker user, and i dont know if a docker version is available.
lets ask @aimc this question.

11

Our official “beta” docker builds are currently broken. We’re working on it. In the meantime, if you’d like to try it out, you’re welcome to use my docker image. It’s not automated, but I try to keep it up to date with AD’s “dev” tree, which is currently the same as the “beta”.

Just use “dlashua/appdaemon:dev” as your image.

I recommend that you backup your current appdaemon.yaml because there are some breaking changes to it.

1 Like
12

ok, that answer does it also :wink:

13

Wooow awesome! Lot of changes, looking forward to try all the new features. I can’t thank you guys enough!

3 Likes
14

Okay, I think we’ve got these use cases covered.

There will be self.ensure_future(coro) to add tasks.
There will be self.run_in_executor(func, *args) to handle sync functions from within async functions

And, of course, if you need a sync function from within a sync function, you just call it like normal.

And all callbacks can be async functions:

def initialize(self):
  self.run_in(self.async_cb, 3)

async def async_cb(self, kwargs):
  await asyncio.sleep(3)
  self.call_service('light/turn_on', blah, blah)
1 Like
15

@karldonteljames- the dev builds were fixed on docker hub. We aren’t doing fixed betas for docker but if you pull the dev branch you will get access to the changes in beta 1 plus whatever stuff we are working on for beta 2.

16

Awesome!!

For my curiosity: under the hood, is AD now async (i.e., all methods are async and sync variants are merely executor jobs)?

17

It’s a little more cunning than that - AD has had an asynch core since 2.0. Until now, the app threads used a function to make a call form thread land into async land. Swiftly added wrappers that check if you are running in a loop or a thread. If it’s a loop we call the functions directly with await, if not we fallback to the old method.

18

Very cunning (and nicely architected)! :+1:

19

The cool part is that you can mix and match Sync and Async in a single app with no need for any config and it will just work.

Only thing I’m still thinking about is how to approach thread allocation in the case where some apps may never need one.

1 Like
20

Is the challenge knowing when that would be dynamically? If so, what about a config parameter for advanced users – if needs_threads is false, the app assumes that everything will run in the event loop?