Announcement: Appdaemon v4.0.0

aimc · January 12, 2020, 7:50pm

Hello everyone, and thank you all for your vast patience. It is my pleasure to announce the official launch of AppDaemon 4.0.0 after a lengthy beta period.

For those of you that have been following along with the Betas, there isn’t a lot new since beta 2, a few bug fixes and a couple of minor changes including adding support for Python 3.8.

Also, be aware that we are planning on removing support for Python 3.6 and earlier soon.

If you are upgrading from AppDaemon 3, there are a number of important breaking changes, please check below for a definitive list and take a look at the previous beta announcements:

As ever, if you have questions feel free to ask here or on our discord channel here:

Happy Automating!

Change Log

4.0.0 (2020-01-12)

Features

Added events for when an app is initialized or terminated
Added event_fire service call
Added production_mode service call
Added list_services api call
Added the ability to fire an event callback only once, using the oneshot flag
Added the ability to use async functions as endpoint callback
Added the ability for input_select to auto-update when the options changes, without need of refreshing the browser page
Added events for when a webscoket client connects and disconnects
Added support for python 3.8

Fixes

Fixed an issue, where when http is disabled in appdaemon.yaml, AD is unable to start
Fixed issue where the user could potentially create entities in admin, global or appdaemon namespaces

Breaking Changes

None

4.0.0 Beta 2 (2019-10-19)

Features

Added a timeout parameter to listen_state() and listen_event() to delete the callback after a pre-determined interval.
Added render_template() handling
global_modules can now be declared in multiple yaml files
It is now possible to inject arbitrary headers in served http content
Updated camera widget now supports streams and token refreshing
Added input_text and input_datetime widgets
Added the ability to control the number of threadpool workers
Each time a new service is registered, a service_registered event is fired, which can be picked up by apps
Added support for async apps
Added authorization to stream as well as command semantics for various functions
Added sequences
Added sequence widget
Added app access to dashboard directory using self.dashboard_dir
List of available dashes is now alphabetically sorted
Changed namespaces implementation to use shelve instead of JSON enabling non JSON-serializable objects to be stored and also potential performance increases - contributed by Robert Schindler <https://github.com/efficiosoft>__
MDI updated to version 4.4.95 - contributed by Roeland Van Lembergen <https://github.com/clayhill>__

Fixes

Fixed a bug in global_modules that caused a exception
Fixed icon bug in weather widget - contributed by Roeland Van Lembergen <https://github.com/clayhill>__

Breaking Changes

timeout is now an official parameter to listen_state() and listen_event(). If you were using timeout in your kwargs section for either you should rename that parameter.
The camera widget has changed parameters - check the docs for details
Moved the log events from global to admin namespace. if listen_log is just used for listening to logs, it shouldn’t matter
If you have used persistent namespaces in the previous beta it is necessary to delete all saved namespaces by removing all files in the namespaces subdirectory under your appdaemon config directory

4.0.0 Beta1 (2019-08-30)

Features

Apps can now use a simplified version of the import statement e.g. import hassapi as hass or import mqttapi as mqtt. The existing import method will continue to work.
Apps can now use multiple plugin APIs with the get_plugin_api() function
Added ADBase superclass for apps that want to use the get_plugin_api() style of coding
Scheduler rewritten to be more efficiant and allow for microsecond resolution
listen_log() now sends AppDaemon system messages and has the option to set a log level.
Bumped aiohttp to v3.4.4
Added callback locking decorators
Rearchitected the work Q to allow App pinning and avoid re-entrant and concurrent code if desired
Implemented multiple worker Ques to avoid Head of Line blocking
API Calls to control app pinning
Added the run_in_thread() api call - with assistance from Odianosen Ejale <https://github.com/Odianosen25>__
reworked log listening functions to be more robust and added the ability to have multiple callbacks per app
Refactored plugin APIs to remove duplication
Moved constrain_days from being Hass only to all app, regardless of plugin used
Added checking for overdue threads
Added error checking for callback signatures
Added app attributes that allows to access AD’s config and apps directories within apps
Added parse_datetime()
run_once(), run_at() and run_daily() now optionally take parse_time() or parse_datetime() style arguments for specifying time
Refactored appdaemon.py for greater readability and easier maintenance
Expanded on the ability to trigger listen_state callbacks immediately using the immediate flag, without need of specifing the new nor duration parameter.
Allowed to make use of attribute when using the immediate flag in listen_state
Added initial version of the Admin Interface
Added User Defined Namespaces
Rewrote logging to include user defined logs and formats
Added a unified http component to handle API, ADMIN and DASBOARD access on a single port
Added startup conditions to the HASS plugin
Added duplicate filtering for logs
Added standalone pidfile functionality
Added the ability to delete an AD app generated entity from any namespace
Added the ability to get the history of entities from HASS database
Added the ability to force a start of the MQTT plugin, even if not connected to broker at startup
Added the ability to set AD’s production_mode from within apps
Added the ability to start, stop, restart and reload apps from either other apps or REST API
Added the ability to register app services
Added sensors for different internal state of AD, that can be read by apps
Added Person widget
Much reworking of docs
Added register_dependency() for dynamic dependencies in apps
Added MQTT support for setting TLS version - contributed by Miguel <https://github.com/mdps>__
Added support for socketio for older tablet devices - inspired by algirdasc <https://github.com/algirdasc>__ and zarya <https://github.com/zarya>__
Added support for default and copy parameters in get_state() api call - contributed by Robert Schindler <https://github.com/efficiosoft>__
added a switch to disable the encoding of every log message to ascii - contributed by Ben Lebherz <https://github.com/benleb>__
Various YAML fixes and refactoring - contributed by Rolf Schäuble <https://github.com/rschaeuble>__
Allow more natural addition of commandline arguments to Docker and allow spaces - contributed by Christoph Roeder <https://github.com/brightdroid>__
Allowed for subscribing to MQTT events using wildcards. e.g. homeassistant/# - contributed by Odianosen Ejale <https://github.com/Odianosen25>__
Allow to specify a MQTT message to be sent when AD shutsdown cleanly e.g. offline
MQTT Retain setting for birth and will messages - contributed by Clifford W. Hansen <https://github.com/cliffordwhansen>__
Added Note on long lived tokens for Docker users - contributed by Bob Anderson <https://github.com/rwa>__
Documentation fixes - contributed by Johann Schmitz <https://github.com/ercpe>__
Documentation fixes - contributed by Brendon Baumgartner <https://github.com/bbrendon>__
Documentation fixes - contributed by Quentin Favrie <https://github.com/tseho>__
Documentation fixes, updating and cleaning - contributed by Humberto Rodríguez A. <https://github.com/rhumbertgz>__
Added the ability to set title 2 as friendly name in widgets - contributed by Radim <https://github.com/rds76>__
Added the ability to listen to state_change events, without using listen_state() - contributed by Thomas Delaet <https://github.com/thomasdelaet>__
APIAI updated to dialog flow - contributed by engrbm87 <https://github.com/engrbm87>__

Fixes

Fixes to listen_state() oneshot function
Fixes to listen_state() oneshot function when duration is used
Fixes to listen_state() function when it fires even when new and old states are same
Fixed an issue causing incorrect busy thread counts when app callbacks had exceptions
Fixed an issue of when MQTT Plugin not connected to broker, and it holds up AD startup
Fix to Forcast min/max in weather widget - contributed by adipose <https://github.com/adipose>__
Fix climate widget docs - contributed by Rene Tode <https://github.com/ReneTode>__
Fix to harmonize units vs unit - contributed by Rene Tode <https://github.com/ReneTode>__
Added missing import in sound.py example - contributed by cclaus <https://github.com/cclauss>__
Fix for run_once() - contributed by engrbm87 <https://github.com/engrbm87>__
Fix for onclick not working on IE11 - contributed by jgrieger1 <https://github.com/jgrieger1>__
Fixed issue of AppDaemon loading all .yaml files, even those starting with a . which are hidden or binary files. Contributed by fhirschmann <https://github.com/fhirschmann>__
Fix for error generated when a none existent schedule timer is passed to info_timer
Fix for log_type flag in listen_log callback
Relative paths for appdaemon’s config directory now work corrcetly
Fix to Dialogflow after format changes
MQTT fix to subscribing using wildcards - contributed by Daniel Lashua <https://github.com/dlashua>__

Breaking Changes

appapi.py has been renamed to adbase.py, and the contained superclass ha been renamed from AppDaemon to ADBase. This should only be a breaking change if you were using unpublished interfaces!
Time travel semantics have changed to support faster scheduling.
plugin_started and plugin_stopped now go to the appropriate namespace for the plugin and are no longer global
Apps are no longer concurrent or re-entrant by default. This is most likely a good thing.
Changed the signature of listen_log() callbacks
cancel_listen_log() now requires a handle supplied by the initial listen_log()
Removed Daemonize support - please use sysctl instead
set_app_state() is deprecated - use set_state() instead and it should do the right thing
dash_compile_on_start now defaults to true
The log section of appdaemon.yaml has been deprecated and must be replaced by the new logs section which has a different format to allow for user defined logs and greater flexibility in formatting etc.
API no longer has a separate port, all access is configured via the new unified http component
API has its own top level configuration section
Some dashboard parameters moved to the HTTP section and renamed
dash_compile_on_start renamed to compile_on_start
dash_force_compile renamed to force_compile
Due to the new log parameter to allow apps to use user defined logs, any previous parameters named log should be renamed
Due to a fix for info_timer, this function can now return None if the timer handle is invalid
As a result of a change in the way AD auto generates MQTT client status topic, if not defined previously the new topic needs to be used
In the appdaemon configuration section, latitude, longitude, elevation and timezone are now mandatory
MQTT client status api change from clientConnected to is_client_connected

Robban · January 12, 2020, 8:14pm

Woohoo Awesome!

Burningstone · January 12, 2020, 9:01pm

Thanks to all the contributors for the great work!

bachya · January 12, 2020, 9:16pm

Thank you, AD team!

Robban · January 12, 2020, 9:43pm

One breaking change not documented: get_state now take entity_id rather than entity

balk77 · January 12, 2020, 9:44pm

What is the difference?

ReneTode · January 12, 2020, 10:02pm

@balk77 that will matter if you use:

self.get_state(entity = “light.a”)

but most people just use

self.get_state(“light.a”)

so its a minor breaking change, but thx. for mentioning it @Robban

sj3fk3 · January 15, 2020, 11:40am

For some reason this does not work for me… The whole “path” works fine, just not the short annotation and it does not help that I don’t understand how this while module sys path thingy works.

Should it work in the “new” 4.0.0. am I doing something wrong? My dev environment is a virtual env “prod” runs in a docker.

Burningstone · January 15, 2020, 11:46am

I’m using the “new style” for AppDaemon 4 apps as suggested by @ReneTode in this post:

ih8gates · January 15, 2020, 12:22pm

I’m brand new to Appdaemon - just started using it last week.

If I want to move from v3 to v4 on Hassio, is it just a matter of uninstalling the old addon and installing the new? (And checking breaking changes)

Burningstone · January 15, 2020, 12:27pm

I don’t use Hass.io, but there is an explanation on how to upgrade in the Github of the addon here.

Bieniu · January 15, 2020, 12:42pm

I have this error:

2020-01-15 13:38:06.348176 WARNING dune-activity: Traceback (most recent call last):
  File "/srv/appdaemon/lib/python3.7/site-packages/appdaemon/app_management.py", line 820, in check_app_updates
    await self.init_object(app)
  File "/srv/appdaemon/lib/python3.7/site-packages/appdaemon/app_management.py", line 280, in init_object
    self.AD, name, self.AD.logging, app_args, self.AD.config, self.app_config, self.AD.global_vars
  File "/srv/appdaemon/lib/python3.7/site-packages/appdaemon/plugins/hass/hassapi.py", line 46, in __init__
    adbase.ADBase.__init__(self, ad, name, logging, args, config, app_config, global_vars)
  File "/srv/appdaemon/lib/python3.7/site-packages/appdaemon/adbase.py", line 75, in __init__
    self.dashboard_dir = self.AD.http.dashboard_dir
AttributeError: 'NoneType' object has no attribute 'dashboard_dir'

2020-01-15 13:38:06.350325 WARNING dune-activity: ------------------------------------------------------------
2020-01-15 13:38:06.352556 WARNING AppDaemon: Logged an error to /home/homeassistant/.homeassistant/appdaemon.log
2020-01-15 13:38:06.357427 WARNING AppDaemon: Unable to find module dune-activity - initialize() skipped

for all ma apps.

AppDaemon config:

logs:
  main_log:
    filename: "/home/homeassistant/.homeassistant/appdaemon.log"
    log_size: 5120
    log_generations: 0
  error_log:
    filename: "/home/homeassistant/.homeassistant/appdaemon.log"
    log_size: 5120
    log_generations: 0
appdaemon:
  threads: 10
  app_dir: "/home/homeassistant/.homeassistant/appdaemon/apps"
  latitude: !secret home_latitude
  longitude: !secret home_longitude
  elevation: 97
  time_zone: "Europe/Warsaw"
  plugins:
    HASS:
      type: hass
      ha_url: !secret local_url
      token: !secret appdaemon_token

I don’t use dashboard. Any solution for that?

Odianosen25 · January 15, 2020, 1:35pm

Hello @Bieniu,

This is a bug, so just add the http component https://appdaemon.readthedocs.io/en/latest/CONFIGURE.html#configuring-the-http-component to fix it for now. Will be fixed in the next release.

Regards

faspina · January 15, 2020, 3:15pm

I am trying to upgrade to 4.0.0 running docker

Getting:

‘File “/usr/local/lib/python3.8/site-packages/appdaemon/app_management.py”, line 145, in initialize_app
await utils.run_in_executor(self, init)
File “/usr/local/lib/python3.8/site-packages/appdaemon/utils.py”, line 276, in run_in_executor
response = future.result()
File “/usr/local/lib/python3.8/concurrent/futures/thread.py”, line 57, in run
result = self.fn(*self.args, **self.kwargs)
File “/conf/apps/sensorage.py”, line 20, in initialize
self.handle = self.run_every(self.age_check, self.datetime(), 5 * 60)
File “/usr/local/lib/python3.8/site-packages/appdaemon/utils.py”, line 191, in inner_sync_wrapper
f = run_coroutine_threadsafe(self, coro(self, *args, **kwargs))
File “/usr/local/lib/python3.8/site-packages/appdaemon/utils.py”, line 285, in run_coroutine_threadsafe
result = future.result(self.AD.internal_function_timeout)
File “/usr/local/lib/python3.8/concurrent/futures/_base.py”, line 439, in result
return self.__get_result()
File “/usr/local/lib/python3.8/concurrent/futures/_base.py”, line 388, in __get_result
raise self._exception
File “/usr/local/lib/python3.8/site-packages/appdaemon/adapi.py”, line 2476, in run_every
raise ValueError(“start cannot be in the past”)
ValueError: start cannot be in the past’

On this part of the code.

class SensorAge(hass.Hass):
def initialize(self):
self.handle = self.run_every(self.age_check, self.datetime(), 5 * 60)

Any ideas

ReneTode · January 15, 2020, 3:45pm

self.run_every always needed a time in the future.
so using it like that was always tricky, when your system was a little busy it would have thrown an error.
add a second to self.datetime() to make sure its a time in the future

datetime.timedelta(seconds=1)

faspina · January 15, 2020, 3:47pm

Yep, I just figured that out.

sbmlat · January 28, 2020, 1:15am

I just upgraded to the latest version. I am running AD in a docker container on unRAID.

I found this error from AD container log file.

Any ideas?

Traceback (most recent call last):
File "/usr/local/bin/appdaemon", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/site-packages/appdaemon/__main__.py", line 384, in main
admain.main()
File "/usr/local/lib/python3.8/site-packages/appdaemon/__main__.py", line 330, in main
self.logging = logging.Logging(logs, args.debug)
File "/usr/local/lib/python3.8/site-packages/appdaemon/logging.py", line 245, in __init__
self.config[log][arg] = config[log][arg]
TypeError: string indices must be integers

berniebl · January 28, 2020, 1:26am

I just upgraded as well, and I am having pretty much the same issues.
I commented out the “log:” section and if I remember correctly, this part of the error message went away.

Now I am getting a bunch of these:
ERROR AppDaemon: time_zone not specified in appdaemon.cfg
ERROR AppDaemon: latitude not specified in appdaemon.cfg
ERROR AppDaemon: longitude not specified in appdaemon.cfg
ERROR AppDaemon: elevation not specified in appdaemon.cfg

I need to go back and try to read all the breaking changes. I am surprised to read that for a lot of people the upgrade went smoothly. I don’t have any apps running except Hello World and 1 dashboard.

sbmlat · January 28, 2020, 1:31am

Yes, you need to add these entries to appdaemon.yaml file.

appdaemon:
  latitude: !secret lati
  longitude: !secret long
  elevation: 224
  timezone: America/Chicago

Also update ‘log’ to ‘logs’

logs:
  logfile: /conf/logs/appdaemon.log
  errorfile: /conf/logs/appdaemonerr.log

berniebl · January 28, 2020, 1:44am

Thanks for pointing this out, it will save me some searching.
One question if you don’t mind. The path for logs … is it relative to the directory where appdaemon directory? or is conf/ the appdaemon directory.

Meaning either:

appdaemon/conf/logs
or
appdaemon/logs