Announcement: Appdaemon v4.0.0

Hello everyone, and thank you all for your vast patience. It is my pleasure to announce the official launch of AppDaemon 4.0.0 after a lengthy beta period.

For those of you that have been following along with the Betas, there isn’t a lot new since beta 2, a few bug fixes and a couple of minor changes including adding support for Python 3.8.

Also, be aware that we are planning on removing support for Python 3.6 and earlier soon.

If you are upgrading from AppDaemon 3, there are a number of important breaking changes, please check below for a definitive list and take a look at the previous beta announcements:


As ever, if you have questions feel free to ask here or on our discord channel here:

Happy Automating!

Change Log

4.0.0 (2020-01-12)

Features

  • Added events for when an app is initialized or terminated
  • Added event_fire service call
  • Added production_mode service call
  • Added list_services api call
  • Added the ability to fire an event callback only once, using the oneshot flag
  • Added the ability to use async functions as endpoint callback
  • Added the ability for input_select to auto-update when the options changes, without need of refreshing the browser page
  • Added events for when a webscoket client connects and disconnects
  • Added support for python 3.8

Fixes

  • Fixed an issue, where when http is disabled in appdaemon.yaml, AD is unable to start
  • Fixed issue where the user could potentially create entities in admin, global or appdaemon namespaces

Breaking Changes

None

4.0.0 Beta 2 (2019-10-19)

Features

  • Added a timeout parameter to listen_state() and listen_event() to delete the callback after a pre-determined interval.
  • Added render_template() handling
  • global_modules can now be declared in multiple yaml files
  • It is now possible to inject arbitrary headers in served http content
  • Updated camera widget now supports streams and token refreshing
  • Added input_text and input_datetime widgets
  • Added the ability to control the number of threadpool workers
  • Each time a new service is registered, a service_registered event is fired, which can be picked up by apps
  • Added support for async apps
  • Added authorization to stream as well as command semantics for various functions
  • Added sequences
  • Added sequence widget
  • Added app access to dashboard directory using self.dashboard_dir
  • List of available dashes is now alphabetically sorted
  • Changed namespaces implementation to use shelve instead of JSON enabling non JSON-serializable objects to be stored and also potential performance increases - contributed by Robert Schindler <https://github.com/efficiosoft>__
  • MDI updated to version 4.4.95 - contributed by Roeland Van Lembergen <https://github.com/clayhill>__

Fixes

  • Fixed a bug in global_modules that caused a exception
  • Fixed icon bug in weather widget - contributed by Roeland Van Lembergen <https://github.com/clayhill>__

Breaking Changes

  • timeout is now an official parameter to listen_state() and listen_event(). If you were using timeout in your kwargs section for either you should rename that parameter.
  • The camera widget has changed parameters - check the docs for details
  • Moved the log events from global to admin namespace. if listen_log is just used for listening to logs, it shouldn’t matter
  • If you have used persistent namespaces in the previous beta it is necessary to delete all saved namespaces by removing all files in the namespaces subdirectory under your appdaemon config directory

4.0.0 Beta1 (2019-08-30)

Features

  • Apps can now use a simplified version of the import statement e.g. import hassapi as hass or import mqttapi as mqtt. The existing import method will continue to work.
  • Apps can now use multiple plugin APIs with the get_plugin_api() function
  • Added ADBase superclass for apps that want to use the get_plugin_api() style of coding
  • Scheduler rewritten to be more efficiant and allow for microsecond resolution
  • listen_log() now sends AppDaemon system messages and has the option to set a log level.
  • Bumped aiohttp to v3.4.4
  • Added callback locking decorators
  • Rearchitected the work Q to allow App pinning and avoid re-entrant and concurrent code if desired
  • Implemented multiple worker Ques to avoid Head of Line blocking
  • API Calls to control app pinning
  • Added the run_in_thread() api call - with assistance from Odianosen Ejale <https://github.com/Odianosen25>__
  • reworked log listening functions to be more robust and added the ability to have multiple callbacks per app
  • Refactored plugin APIs to remove duplication
  • Moved constrain_days from being Hass only to all app, regardless of plugin used
  • Added checking for overdue threads
  • Added error checking for callback signatures
  • Added app attributes that allows to access AD’s config and apps directories within apps
  • Added parse_datetime()
  • run_once(), run_at() and run_daily() now optionally take parse_time() or parse_datetime() style arguments for specifying time
  • Refactored appdaemon.py for greater readability and easier maintenance
  • Expanded on the ability to trigger listen_state callbacks immediately using the immediate flag, without need of specifing the new nor duration parameter.
  • Allowed to make use of attribute when using the immediate flag in listen_state
  • Added initial version of the Admin Interface
  • Added User Defined Namespaces
  • Rewrote logging to include user defined logs and formats
  • Added a unified http component to handle API, ADMIN and DASBOARD access on a single port
  • Added startup conditions to the HASS plugin
  • Added duplicate filtering for logs
  • Added standalone pidfile functionality
  • Added the ability to delete an AD app generated entity from any namespace
  • Added the ability to get the history of entities from HASS database
  • Added the ability to force a start of the MQTT plugin, even if not connected to broker at startup
  • Added the ability to set AD’s production_mode from within apps
  • Added the ability to start, stop, restart and reload apps from either other apps or REST API
  • Added the ability to register app services
  • Added sensors for different internal state of AD, that can be read by apps
  • Added Person widget
  • Much reworking of docs
  • Added register_dependency() for dynamic dependencies in apps
  • Added MQTT support for setting TLS version - contributed by Miguel <https://github.com/mdps>__
  • Added support for socketio for older tablet devices - inspired by algirdasc <https://github.com/algirdasc>__ and zarya <https://github.com/zarya>__
  • Added support for default and copy parameters in get_state() api call - contributed by Robert Schindler <https://github.com/efficiosoft>__
  • added a switch to disable the encoding of every log message to ascii - contributed by Ben Lebherz <https://github.com/benleb>__
  • Various YAML fixes and refactoring - contributed by Rolf Schäuble <https://github.com/rschaeuble>__
  • Allow more natural addition of commandline arguments to Docker and allow spaces - contributed by Christoph Roeder <https://github.com/brightdroid>__
  • Allowed for subscribing to MQTT events using wildcards. e.g. homeassistant/# - contributed by Odianosen Ejale <https://github.com/Odianosen25>__
  • Allow to specify a MQTT message to be sent when AD shutsdown cleanly e.g. offline
  • MQTT Retain setting for birth and will messages - contributed by Clifford W. Hansen <https://github.com/cliffordwhansen>__
  • Added Note on long lived tokens for Docker users - contributed by Bob Anderson <https://github.com/rwa>__
  • Documentation fixes - contributed by Johann Schmitz <https://github.com/ercpe>__
  • Documentation fixes - contributed by Brendon Baumgartner <https://github.com/bbrendon>__
  • Documentation fixes - contributed by Quentin Favrie <https://github.com/tseho>__
  • Documentation fixes, updating and cleaning - contributed by Humberto Rodríguez A. <https://github.com/rhumbertgz>__
  • Added the ability to set title 2 as friendly name in widgets - contributed by Radim <https://github.com/rds76>__
  • Added the ability to listen to state_change events, without using listen_state() - contributed by Thomas Delaet <https://github.com/thomasdelaet>__
  • APIAI updated to dialog flow - contributed by engrbm87 <https://github.com/engrbm87>__

Fixes

  • Fixes to listen_state() oneshot function
  • Fixes to listen_state() oneshot function when duration is used
  • Fixes to listen_state() function when it fires even when new and old states are same
  • Fixed an issue causing incorrect busy thread counts when app callbacks had exceptions
  • Fixed an issue of when MQTT Plugin not connected to broker, and it holds up AD startup
  • Fix to Forcast min/max in weather widget - contributed by adipose <https://github.com/adipose>__
  • Fix climate widget docs - contributed by Rene Tode <https://github.com/ReneTode>__
  • Fix to harmonize units vs unit - contributed by Rene Tode <https://github.com/ReneTode>__
  • Added missing import in sound.py example - contributed by cclaus <https://github.com/cclauss>__
  • Fix for run_once() - contributed by engrbm87 <https://github.com/engrbm87>__
  • Fix for onclick not working on IE11 - contributed by jgrieger1 <https://github.com/jgrieger1>__
  • Fixed issue of AppDaemon loading all .yaml files, even those starting with a . which are hidden or binary files. Contributed by fhirschmann <https://github.com/fhirschmann>__
  • Fix for error generated when a none existent schedule timer is passed to info_timer
  • Fix for log_type flag in listen_log callback
  • Relative paths for appdaemon’s config directory now work corrcetly
  • Fix to Dialogflow after format changes
  • MQTT fix to subscribing using wildcards - contributed by Daniel Lashua <https://github.com/dlashua>__

Breaking Changes

  • appapi.py has been renamed to adbase.py, and the contained superclass ha been renamed from AppDaemon to ADBase. This should only be a breaking change if you were using unpublished interfaces!
  • Time travel semantics have changed to support faster scheduling.
  • plugin_started and plugin_stopped now go to the appropriate namespace for the plugin and are no longer global
  • Apps are no longer concurrent or re-entrant by default. This is most likely a good thing.
  • Changed the signature of listen_log() callbacks
  • cancel_listen_log() now requires a handle supplied by the initial listen_log()
  • Removed Daemonize support - please use sysctl instead
  • set_app_state() is deprecated - use set_state() instead and it should do the right thing
  • dash_compile_on_start now defaults to true
  • The log section of appdaemon.yaml has been deprecated and must be replaced by the new logs section which has a different format to allow for user defined logs and greater flexibility in formatting etc.
  • API no longer has a separate port, all access is configured via the new unified http component
  • API has its own top level configuration section
  • Some dashboard parameters moved to the HTTP section and renamed
  • dash_compile_on_start renamed to compile_on_start
  • dash_force_compile renamed to force_compile
  • Due to the new log parameter to allow apps to use user defined logs, any previous parameters named log should be renamed
  • Due to a fix for info_timer, this function can now return None if the timer handle is invalid
  • As a result of a change in the way AD auto generates MQTT client status topic, if not defined previously the new topic needs to be used
  • In the appdaemon configuration section, latitude, longitude, elevation and timezone are now mandatory
  • MQTT client status api change from clientConnected to is_client_connected
12 Likes

Woohoo Awesome!

Thanks to all the contributors for the great work!

Thank you, AD team!

One breaking change not documented: get_state now take entity_id rather than entity

1 Like

What is the difference?

@balk77 that will matter if you use:

self.get_state(entity = “light.a”)

but most people just use

self.get_state(“light.a”)

so its a minor breaking change, but thx. for mentioning it @Robban

1 Like

For some reason this does not work for me… The whole “path” works fine, just not the short annotation and it does not help that I don’t understand how this while module sys path thingy works.

Should it work in the “new” 4.0.0. am I doing something wrong? My dev environment is a virtual env “prod” runs in a docker.

I’m using the “new style” for AppDaemon 4 apps as suggested by @ReneTode in this post:

I’m brand new to Appdaemon - just started using it last week.

If I want to move from v3 to v4 on Hassio, is it just a matter of uninstalling the old addon and installing the new? (And checking breaking changes)

I don’t use Hass.io, but there is an explanation on how to upgrade in the Github of the addon here.

2 Likes

I have this error:

2020-01-15 13:38:06.348176 WARNING dune-activity: Traceback (most recent call last):
  File "/srv/appdaemon/lib/python3.7/site-packages/appdaemon/app_management.py", line 820, in check_app_updates
    await self.init_object(app)
  File "/srv/appdaemon/lib/python3.7/site-packages/appdaemon/app_management.py", line 280, in init_object
    self.AD, name, self.AD.logging, app_args, self.AD.config, self.app_config, self.AD.global_vars
  File "/srv/appdaemon/lib/python3.7/site-packages/appdaemon/plugins/hass/hassapi.py", line 46, in __init__
    adbase.ADBase.__init__(self, ad, name, logging, args, config, app_config, global_vars)
  File "/srv/appdaemon/lib/python3.7/site-packages/appdaemon/adbase.py", line 75, in __init__
    self.dashboard_dir = self.AD.http.dashboard_dir
AttributeError: 'NoneType' object has no attribute 'dashboard_dir'

2020-01-15 13:38:06.350325 WARNING dune-activity: ------------------------------------------------------------
2020-01-15 13:38:06.352556 WARNING AppDaemon: Logged an error to /home/homeassistant/.homeassistant/appdaemon.log
2020-01-15 13:38:06.357427 WARNING AppDaemon: Unable to find module dune-activity - initialize() skipped

for all ma apps.

AppDaemon config:

logs:
  main_log:
    filename: "/home/homeassistant/.homeassistant/appdaemon.log"
    log_size: 5120
    log_generations: 0
  error_log:
    filename: "/home/homeassistant/.homeassistant/appdaemon.log"
    log_size: 5120
    log_generations: 0
appdaemon:
  threads: 10
  app_dir: "/home/homeassistant/.homeassistant/appdaemon/apps"
  latitude: !secret home_latitude
  longitude: !secret home_longitude
  elevation: 97
  time_zone: "Europe/Warsaw"
  plugins:
    HASS:
      type: hass
      ha_url: !secret local_url
      token: !secret appdaemon_token

I don’t use dashboard. Any solution for that?

1 Like

Hello @Bieniu,

This is a bug, so just add the http component https://appdaemon.readthedocs.io/en/latest/CONFIGURE.html#configuring-the-http-component to fix it for now. Will be fixed in the next release.

Regards

3 Likes

I am trying to upgrade to 4.0.0 running docker

Getting:

‘File “/usr/local/lib/python3.8/site-packages/appdaemon/app_management.py”, line 145, in initialize_app
await utils.run_in_executor(self, init)
File “/usr/local/lib/python3.8/site-packages/appdaemon/utils.py”, line 276, in run_in_executor
response = future.result()
File “/usr/local/lib/python3.8/concurrent/futures/thread.py”, line 57, in run
result = self.fn(*self.args, **self.kwargs)
File “/conf/apps/sensorage.py”, line 20, in initialize
self.handle = self.run_every(self.age_check, self.datetime(), 5 * 60)
File “/usr/local/lib/python3.8/site-packages/appdaemon/utils.py”, line 191, in inner_sync_wrapper
f = run_coroutine_threadsafe(self, coro(self, *args, **kwargs))
File “/usr/local/lib/python3.8/site-packages/appdaemon/utils.py”, line 285, in run_coroutine_threadsafe
result = future.result(self.AD.internal_function_timeout)
File “/usr/local/lib/python3.8/concurrent/futures/_base.py”, line 439, in result
return self.__get_result()
File “/usr/local/lib/python3.8/concurrent/futures/_base.py”, line 388, in __get_result
raise self._exception
File “/usr/local/lib/python3.8/site-packages/appdaemon/adapi.py”, line 2476, in run_every
raise ValueError(“start cannot be in the past”)
ValueError: start cannot be in the past’

On this part of the code.

class SensorAge(hass.Hass):
def initialize(self):
self.handle = self.run_every(self.age_check, self.datetime(), 5 * 60)

Any ideas

self.run_every always needed a time in the future.
so using it like that was always tricky, when your system was a little busy it would have thrown an error.
add a second to self.datetime() to make sure its a time in the future

  • datetime.timedelta(seconds=1)

Yep, I just figured that out.

I just upgraded to the latest version. I am running AD in a docker container on unRAID.

I found this error from AD container log file.

Any ideas?

Traceback (most recent call last):
File "/usr/local/bin/appdaemon", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/site-packages/appdaemon/__main__.py", line 384, in main
admain.main()
File "/usr/local/lib/python3.8/site-packages/appdaemon/__main__.py", line 330, in main
self.logging = logging.Logging(logs, args.debug)
File "/usr/local/lib/python3.8/site-packages/appdaemon/logging.py", line 245, in __init__
self.config[log][arg] = config[log][arg]
TypeError: string indices must be integers

I just upgraded as well, and I am having pretty much the same issues.
I commented out the “log:” section and if I remember correctly, this part of the error message went away.

Now I am getting a bunch of these:
ERROR AppDaemon: time_zone not specified in appdaemon.cfg
ERROR AppDaemon: latitude not specified in appdaemon.cfg
ERROR AppDaemon: longitude not specified in appdaemon.cfg
ERROR AppDaemon: elevation not specified in appdaemon.cfg

I need to go back and try to read all the breaking changes. I am surprised to read that for a lot of people the upgrade went smoothly. I don’t have any apps running except Hello World and 1 dashboard.

Yes, you need to add these entries to appdaemon.yaml file.

appdaemon:
  latitude: !secret lati
  longitude: !secret long
  elevation: 224
  timezone: America/Chicago

Also update ‘log’ to ‘logs’

logs:
  logfile: /conf/logs/appdaemon.log
  errorfile: /conf/logs/appdaemonerr.log

Thanks for pointing this out, it will save me some searching.
One question if you don’t mind. The path for logs … is it relative to the directory where appdaemon directory? or is conf/ the appdaemon directory.

Meaning either:

  1. appdaemon/conf/logs
    or
  2. appdaemon/logs