Newbie - Log files

sbmlat · October 3, 2017, 2:18am

Hi,

I recently jumped into AppDaemon bandwagon. Currently I am running HA in venv on Pi3. I converted a couple of my existing automations using AppDaemon but I got a lot of newbie questions.

Where do these logs write?

ex: self.log(‘TurnOnLightsAtSunset: Sunset triggered’)

I searched in /var/logs but I couldn’t find there.

My config setting:

AppDaemon:
logfile: STDOUT
errorfile: STDERR

I tried to change to a different file but got file access errors. Can I write these logs to existing HA log?

ex:
logfile: /var/log/appdaemon.log
errorfile: /var/log/appdaemonerr.log

How do I send a notification to IFTTT from appdaemon? I tried following snippet (got from the forums only) but that didn’t trigger IFTTT notification.

def send_notification_msg(self, msg):
    self.notify(msg, name=self.args["notify_who"],title="TurnOnLightsAtSunset")
    msg="Porch light and Kitchen lights turned on after sunset - Time now : {}".time.time()
    self.log(msg)

Config setting:

sunset_lights:
module: sunsetlights
class: TurnOnLightsAtSunset
notify_who: ifttt

Thank you

SupahNoob · October 3, 2017, 2:39am

I recently jumped into AppDaemon bandwagon. Currently I am running HA in venv on Pi3. I converted a couple of my existing automations using AppDaemon but I got a lot of newbie questions.

Welcome @sbmlat! Once you get the hang of a bit of python, AppDaemon and the interactions you can create within the program and HASS is REALLY powerful and really awesome! Let us know any time you have questions, there’s tons of users here that will pitch in to help you learn.

I’m going to assume you know absolutely nothing and explain in decent detail, even if you can follow perfectly, others might have the same question.

Where do these logs write?

AppDaemon:
logfile: STDOUT
errorfile: STDERR

I tried to change to a different file but got file access errors. Can I write these logs to existing HA log?

This is SORT of a trick question. The lines in your config are basically saying "print out all logs to the console, and don’t save any files at all. These are meant to be entirely optional and no-nonsense, in that you can see all your output directly in the terminal whenever you run AD. Check out this section of the documentation to get a bit more info on these parameters. You could change these to whatever FULL PATH you’d like. AppDaemon should have no problems writing within the venv, so make sure you pick something within that domain! If you put info.log and err.log in those two fields, they would produce files wherever you start AppDaemon from. /var/log/appdaemon.log would work just fine so long as you have write permissions there.

How do I send a notification to IFTTT from appdaemon? I tried following snippet (got from the forums only) but that didn’t trigger IFTTT notification.

I don’t believe IFTTT has native support from within the AppDaemon notify call. Try calling IFTTT as a service, if you have the component already set up within HomeAssistant.

def send_notification_msg(self, msg):
    msg="Porch light and Kitchen lights turned on after sunset - Time now : {}".time.time()
    self.call_service('ifttt.trigger', event='LightsOn', value1=msg)
    #  self.call_service('ifttt.trigger', event=self.args['notify_who'], value1=msg)
    self.log(msg)

I do not use IFTTT, but this should do you alright. You can trigger the event to be custom per the args you set up, if, for example, you had an event that was the same value as what you set in notify_who … this would change from ifttt to something like notify_sbmlat, for example. You’d need a webhook set up in IFTTT named notify_sbmlat to be able to trigger this.

Hope this helps! Keep us updated on your progress.

-SN

gpbenton · October 3, 2017, 4:54am

I have created a /var/log/appdaemon directory, given my homeassistant user write permissions to that directory, and point my configuration to files in the directory.

I suspect using the same log as homeassistant would result in jumbled log files.

ReneTode · October 3, 2017, 9:43am

with STDOUT and STERR the log and errors go to the screen. running AD as daemon they dissapear.
you need to set it to a place where the user that runs AD has writing access. .
I wouldnt try setting it to the HA log. i think you will get problems also.
you set the message from the notification after! you have send the notification. and how do you call the function?
please share your complete app.

sbmlat · October 3, 2017, 2:12pm

Thank you for the detailed reply.

I guess my brain hit snooze and read only logfile and errorfile and didn’t even focus on STDOUT and STDERR part

I tried to use following locations for store log files but both failed with permission denied error message.

  logfile: /var/log/appdaemon.log
  errorfile: /var/log/appdaemonerr.log
  # logfile: /home/homeassistant/.homeassistant/appdaemon.log
  # errorfile: /home/homeassistant/.homeassistant/appdaemonerr.log

PermissionError: [Errno 13] Permission denied: ‘/var/log/appdaemon.log’

I ran the following command under ‘Pi’ user account.

pi@rpi3-aio:~ $ appdaemon -c /home/homeassistant/.homeassistant/config

I wanted to pass msg to send_notification_msg from different function. Yes, Python is not my strong suit. I am learning this on the fly

def sunset_cb(self, kwargs):
    """Turn on the lights when the correct time hits."""
    self.log('TurnOnLightsAtSunset: Sunset triggered')
    self.turn_on('group.sunset_switches')
    self.log('Porch light and Kitchen lights turned on after sunset')
    # msg="Porch light and Kitchen lights turned on after sunset - Time now : {}".time.time()
    # self.send_notification_msg(msg)

I will test out IFTTT notification and let you know.

sbmlat · October 3, 2017, 2:21pm

I tried your suggestion and still fails.

pi@rpi3-aio:/var/log $ sudo mkdir appdaemon
pi@rpi3-aio:/var/log $ sudo chown homeassistant: appdaemon

pi@rpi3-aio:/var/log $ ls -lt
total 34392
-rw-r----- 1 root adm 8575 Oct 3 09:17 auth.log
-rw-r----- 1 root adm 318712 Oct 3 09:17 syslog
-rw-r----- 1 root adm 81736 Oct 3 09:16 messages
drwxr-xr-x 2 homeassistant homeassistant 4096 Oct 3 09:13 appdaemon

pi@rpi3-aio:~ $ appdaemon -c /home/homeassistant/.homeassistant/config
Traceback (most recent call last):
File “/usr/local/bin/appdaemon”, line 11, in
sys.exit(main())
File “/usr/local/lib/python3.4/dist-packages/appdaemon/admain.py”, line 355, in main
fh = RotatingFileHandler(conf.logfile, maxBytes=log_size, backupCount=log_generations)
File “/usr/lib/python3.4/logging/handlers.py”, line 150, in init
BaseRotatingHandler.init(self, filename, mode, encoding, delay)
File “/usr/lib/python3.4/logging/handlers.py”, line 57, in init
logging.FileHandler.init(self, filename, mode, encoding, delay)
File “/usr/lib/python3.4/logging/init.py”, line 1006, in init
StreamHandler.init(self, self._open())
File “/usr/lib/python3.4/logging/init.py”, line 1030, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding)
PermissionError: [Errno 13] Permission denied: ‘/var/log/appdaemon/appdaemon.log’

ReneTode · October 3, 2017, 3:37pm

my brain hit snooze also
i didnt even see there were already others that replied

you run appdaemon as pi user.
so the logfiles should be in a dir where the pi user has writing acces.
chown homeassistant wouldnt help there. that shout be chown pi

i dont know ifttt also. but you can use notify if you can use notify in HA

i understand that you are learning, thats why i asked for your complete app
if you have a problem in your initialize or in the setup from the app you would never get an notify.

but dont try out any notify or anything else if you have no logs.
then you will never know what and if you are doing something wrong

gpbenton · October 3, 2017, 3:45pm

I got halfway through saying this when you posted . I run appdaemon as homeassistant user, which is where the confusion arose.

ReneTode · October 3, 2017, 3:53pm

i got two topics mixed up then
are you sure you run it as homeassistant?
did you also install it as homeassistant?
are all files in the conf dir owned by homeassistant?
try creating a log dir in your config dir as homeassistant and use that.

sbmlat · October 3, 2017, 6:04pm

I installed appdaemon outside venv (my mistake) and it’s running under ‘pi’ account.

pi@rpi3-aio:/home/homeassistant/.homeassistant/config/logs $ ps aux | grep appdaemon
pi 2717 1.6 3.2 162796 31088 ? Sl Oct02 13:59 /usr/bin/python3 /usr/local/bin/appdaemon -c /home/homeassistant/.homeassistant/config
pi 6351 23.6 2.7 161440 26360 pts/4 Sl+ 12:54 0:04 /usr/bin/python3 /usr/local/bin/appdaemon -c /home/homeassistant/.homeassistant/config
pi 6416 0.0 0.2 4272 1936 pts/3 S+ 12:54 0:00 grep --color=auto appdaemon
pi 12122 1.6 3.1 164848 29992 ? Sl Oct01 39:04 /usr/bin/python3 /usr/local/bin/appdaemon -c /home/homeassistant/.homeassistant/config
homeass+ 12436 1.6 3.1 166384 29660 ? Ssl Sep28 107:15 /usr/bin/python3 /usr/local/bin/appdaemon -c /home/homeassistant/.homeassistant/config
pi 20257 1.6 3.0 163008 29168 ? Sl Sep30 71:08 /usr/bin/python3 /usr/local/bin/appdaemon -c /home/homeassistant/.homeassistant/config

No. They all owned by ‘root’.

pi@rpi3-aio:/home/homeassistant/.homeassistant/config $ sudo mkdir logs
pi@rpi3-aio:/home/homeassistant/.homeassistant/config $ ls -lt
total 20
drwxr-xr-x 2 root root 4096 Oct 3 12:41 logs
-rwxrwxrwx 1 root root 636 Oct 3 09:17 appdaemon.yaml
-rwxrwxrwx 1 root root 714 Oct 1 22:55 apps.yaml
drwxrwxrwx 3 root root 4096 Oct 1 22:33 apps
-rwxrwxrwx 1 root root 18 Sep 28 20:39 secrets.yaml
pi@rpi3-aio:/home/homeassistant/.homeassistant/config $ sudo chown homeassistant: logs
pi@rpi3-aio:/home/homeassistant/.homeassistant/config $ ls -lt
total 20
drwxr-xr-x 2 homeassistant homeassistant 4096 Oct 3 12:41 logs
-rwxrwxrwx 1 root root 636 Oct 3 09:17 appdaemon.yaml
-rwxrwxrwx 1 root root 714 Oct 1 22:55 apps.yaml
drwxrwxrwx 3 root root 4096 Oct 1 22:33 apps
-rwxrwxrwx 1 root root 18 Sep 28 20:39 secrets.yaml

I created new ‘log’ folder and gave the access to ‘homeassistant’ user. But that didn’t work.

Later I found the owner of appdaemon process (from above) then gave access to ‘pi’ user instead then it worked.

pi@rpi3-aio:/home/homeassistant/.homeassistant/config/logs $ ls -lt
total 4
-rw-r–r-- 1 homeassistant homeassistant 2 Oct 3 12:46 appdaemonerr.log
-rw-r–r-- 1 homeassistant homeassistant 0 Oct 3 12:45 appdaemon.log
pi@rpi3-aio:/home/homeassistant/.homeassistant/config/logs $ sudo chown pi: *.log pi@rpi3-aio:/home/homeassistant/.homeassistant/config/logs $ ls -lt
total 4
-rw-r–r-- 1 pi pi 2 Oct 3 12:46 appdaemonerr.log
-rw-r–r-- 1 pi pi 0 Oct 3 12:45 appdaemon.log
pi@rpi3-aio:/home/homeassistant/.homeassistant/config/logs $ tail -f appdaemon.log
2017-10-03 12:56:25.141367 INFO Loading Module: /home/homeassistant/.homeassistant/config/apps/sunsetlights.py
2017-10-03 12:56:25.142798 INFO Loading Object sunset_lights using class TurnOnLightsAtSunset from module sunsetlights
2017-10-03 12:56:25.150122 INFO sunset_lights: Next sunset at 2017-10-03 18:42:26
2017-10-03 12:56:25.151039 INFO Loading Module: /home/homeassistant/.homeassistant/config/apps/bedtimelights.py
2017-10-03 12:56:25.153902 INFO Loading Object bedtime_lights using class TurnOffLightsAtBedtime from module bedtimelights
2017-10-03 12:56:25.163298 INFO bedtime_lights: Today date: 2017-10-03 - Time to trigger: 23:35:00
2017-10-03 12:56:25.164194 INFO App initialization complete
2017-10-03 12:56:25.164824 INFO Dashboards are disabled
2017-10-03 12:56:25.165125 INFO API is disabled
2017-10-03 12:56:25.316014 INFO Connected to Home Assistant 0.54.0

appdaemon.yaml

AppDaemon:
logfile: /home/homeassistant/.homeassistant/config/logs/appdaemon.log
errorfile: /home/homeassistant/.homeassistant/config/logs/appdaemonerr.log
logsize: 100000
log_generations: 3
threads: 10

It was my mistake of not installing ‘appdaemon’ under ‘homeassistant’ user. How can I correct this now?

Thank you

ReneTode · October 3, 2017, 6:52pm

why should you want to correct?
if its working, its working.

yeah you could correct, but you need to uninstall, reinstall as homeassistant user, recreate folders, set the right rights, etc.
i wouldnt bother.

i never installed inside any venv and there is no advantage from keeping them both in the same venv.

sbmlat · October 3, 2017, 8:04pm

I agree. If it ain’t broke, don’t fix it

Now to the next task. Figuring out IFTTT notifications

ReneTode · October 3, 2017, 8:21pm

i guess what is confusing is that you call it notifications.
in HA ifttt isnt a notify component, its a automation component.

so you cant use notify for that.

when i read on the ifttt component website from HA

i find this:

domain ifttt
service trigger
Service Data {“event”: “EventName”, “value1”: “Hello World”}

so i guess the way that @SupahNoob described is the right way.

self.call_service('ifttt.trigger', event='LightsOn', value1=msg)

but it also could be

self.call_service('ifttt/trigger', event='LightsOn', value1=msg)

i would suggest to try it out in a working app. (for instance the hello world app)

sbmlat · October 3, 2017, 11:17pm

My bad. Let me explain bit more. I have existing automation where it sends a message to my phone via IFTTT when the lights are turned on in the house.

Ex:

– 092817 - Moved this logic to appdeamon

alias: Zwave Turn on porch light and kitchen lights after sun sets
initial_state: false
trigger:
platform: sun
event: sunset
offset: ‘+00:10:00’
action:

service: homeassistant.turn_on
data:
entity_id: switch.ge_12722_onoff_relay_switch_switch_3

service: homeassistant.turn_on
data:
entity_id: switch.ge_12722_onoff_relay_switch_switch_4

service: script.notify_me
data: {“value1”:“Porch light and Kitchen lights turned on after sunset”, “value2”:“”}

Btw, after I updated the logic with your suggestion. It worked!

appdaemon for same logic:

“”“Define automations for evening-time lighting.”“”

import appdaemon.appapi as appapi
import time

class TurnOnLightsAtSunset(appapi.AppDaemon):
“”“Define a class to represent the app.”“”

def initialize(self):
    """Initialize."""
    self.log('Next sunset at ' + str(self.sunset()))
    self.run_at_sunset(self.sunset_cb)
    msg="Test IFTTT 2"
    self.send_notification_msg(msg)

def sunset_cb(self, kwargs):
    """Turn on the lights when the correct time hits."""
    self.log('TurnOnLightsAtSunset: Sunset triggered')
    self.turn_on('group.sunset_switches')
    self.log('Porch light and Kitchen lights turned on after sunset')
    # msg="Porch light and Kitchen lights turned on after sunset - Time now : {}".time.time()
    msg="Porch light and Kitchen lights turned on after sunset"
    self.send_notification_msg(msg)

def send_notification_msg(self, msg):
    # msg="Porch light and Kitchen lights turned on after sunset - Time now : {}".time.time()
    self.call_service('ifttt/trigger', event='ShreRam_Home', value1=msg)
    self.log(msg)

Here is the screenshot:

SupahNoob · October 4, 2017, 2:48am

Awesome! Glad you got that working so quickly. I’m excited to see what other automations you post here. Welcome @sbmlat!

sbmlat · October 5, 2017, 2:18am

Thank you. I am still learning.

Btw, today lights didn’t turn on and checked the appdaemon status. I found following errors but not related to the class which turns on lights at sunset.

pi@rpi3-aio:/home/homeassistant/.homeassistant/config/logs $ sudo systemctl status [email protected] -l
● [email protected] - AppDaemon service for homeassistant
Loaded: loaded (/etc/systemd/system/[email protected]; enabled)
Active: active (running) since Thu 2017-09-28 23:49:07 CDT; 5 days ago
Main PID: 12436 (appdaemon)
CGroup: /system.slice/system-appdaemon.slice/[email protected]
└─12436 /usr/bin/python3 /usr/local/bin/appdaemon -c /home/homeassistant/.homeassistant/config

Oct 04 20:51:49 rpi3-aio appdaemon[12436]: 2017-10-04 20:51:49.907213 INFO Got initial state
Oct 04 20:51:49 rpi3-aio appdaemon[12436]: 2017-10-04 20:51:49.910417 ERROR Unable to resolve dependencies due to incorrect or circular references
Oct 04 20:51:49 rpi3-aio appdaemon[12436]: 2017-10-04 20:51:49.912006 ERROR The following modules have unresolved dependencies:
Oct 04 20:51:49 rpi3-aio appdaemon[12436]: 2017-10-04 20:51:49.913672 ERROR bedtimelights
Oct 04 20:51:49 rpi3-aio appdaemon[12436]: 2017-10-04 20:51:49.915189 WARNING Disconnected from Home Assistant, retrying in 5 seconds
Oct 04 20:51:55 rpi3-aio appdaemon[12436]: 2017-10-04 20:51:55.432986 INFO Got initial state
Oct 04 20:51:55 rpi3-aio appdaemon[12436]: 2017-10-04 20:51:55.436108 ERROR Unable to resolve dependencies due to incorrect or circular references
Oct 04 20:51:55 rpi3-aio appdaemon[12436]: 2017-10-04 20:51:55.437636 ERROR The following modules have unresolved dependencies:
Oct 04 20:51:55 rpi3-aio appdaemon[12436]: 2017-10-04 20:51:55.439134 ERROR bedtimelights
Oct 04 20:51:55 rpi3-aio appdaemon[12436]: 2017-10-04 20:51:55.440590 WARNING Disconnected from Home Assistant, retrying in 5 seconds

When I manually I invoke appdeamon then there were no errors in the log file.

pi@rpi3-aio:/home/homeassistant/.homeassistant/config/logs $ tail -f appdaemon.log
2017-10-04 20:51:24.380463 INFO Loading Module: /home/homeassistant/.homeassistant/config/apps/sunsetlights.py
2017-10-04 20:51:24.383186 INFO Loading Object sunset_lights using class TurnOnLightsAtSunset from module sunsetlights
2017-10-04 20:51:24.391038 INFO sunset_lights: Next sunset at 2017-10-05 18:38:50
2017-10-04 20:51:24.392067 INFO Loading Module: /home/homeassistant/.homeassistant/config/apps/bedtimelights.py
2017-10-04 20:51:24.395187 INFO Loading Object bedtime_lights using class TurnOffLightsAtBedtime from module bedtimelights
2017-10-04 20:51:24.402503 INFO bedtime_lights: Today date: 2017-10-04 - Time to trigger: 23:35:00
2017-10-04 20:51:24.403425 INFO App initialization complete
2017-10-04 20:51:24.404076 INFO Dashboards are disabled
2017-10-04 20:51:24.404390 INFO API is disabled
2017-10-04 20:51:24.544075 INFO Connected to Home Assistant 0.54.0

There was no error in appdaemonerr.log file for today.

SupahNoob · October 5, 2017, 2:49am

Can you post your full code for bedtimelights.py and your apps.yaml file? If there is any sensitive information there, you can replace it with dummy names. You have a circular dependency, which means 2 files are trying to depend on each other.

sbmlat · October 5, 2017, 3:20am

Here are the files.

apps.yaml:

hello_world:
  module: hello
  class: HelloWorld
sunset_lights:
  # module - the name of the module (without the .py) that contains the class to be used for this App
  module: sunsetlights
  # class - the name of the class as defined within the module for the APPs code
  class: TurnOnLightsAtSunset
  # ifttt.trigger: ifttt
bedtime_lights:
  module: bedtimelights
  class: TurnOffLightsAtBedtime
  # constrain_start_time: '20:45:00'
  # notify_who: ifttt
  # dependencies: utils
utils:
  module: utils
  class: utilities

bedtimelights.py

import appdaemon.appapi as appapi
# import time
import datetime

class TurnOffLightsAtBedtime(appapi.AppDaemon):
    """Define a class to represent the app."""

    def initialize(self):
        time = datetime.time(23, 35, 0)
        today = datetime.date.today()
        self.log('Today date: ' + str(today) + ' - Time to trigger: ' + str(time))
        self.run_daily(self.run_daily_callback, time)

    def run_daily_callback(self, kwargs):
        self.turn_off("group.zwave_switches")

        msg="Good night. All lights turned off. \n Check the doors. Sleep tight \n"
        self.send_notification_msg(msg)

    def send_notification_msg(self, msg):
        self.call_service('ifttt/trigger', event='ShreRam_Home', value1=msg)

    #  add parameters  - one for entityid, second for time
    # EX: turn off first group lights at 9:30 pm and second group lights at 11:35 pm

Another dummy question.

How do I gracefully close appdeamon process when I manually invoke from command line?

pi@rpi3-aio:~ $ appdaemon -c /home/homeassistant/.homeassistant/config

Usually, I hit Ctrl + C to break.

I notice there were multiple appdeamon instances. I think this is causing notifications (via IFTTT notify) to send multiple times.

pi@rpi3-aio:/home/homeassistant/.homeassistant/config/logs $ ps aux | grep appdaemon
pi        2717  1.7  3.3 163188 31716 ?        Sl   Oct02  48:52 /usr/bin/python3 /usr/local/bin/appdaemon -c /home/homeassistant/.homeassistant/config
pi        3085  0.0  0.2   4272  2044 pts/4    S+   22:08   0:00 grep --color=auto appdaemon
pi       12122  1.6  3.3 164220 31240 ?        Sl   Oct01  73:53 /usr/bin/python3 /usr/local/bin/appdaemon -c /home/homeassistant/.homeassistant/config
homeass+ 12436  1.6  3.2 165616 30448 ?        Ssl  Sep28 143:16 /usr/bin/python3 /usr/local/bin/appdaemon -c /home/homeassistant/.homeassistant/config
pi       20257  1.6  3.1 163264 29784 ?        Sl   Sep30 106:33 /usr/bin/python3 /usr/local/bin/appdaemon -c /home/homeassistant/.homeassistant/config
pi       29114  1.7  3.1 163748 30236 pts/3    Sl+  20:51   1:20 /usr/bin/python3 /usr/local/bin/appdaemon -c /home/homeassistant/.homeassistant/config

pi@rpi3-aio:/home/homeassistant/.homeassistant/config/logs $ kill -9 20257

ReneTode · October 5, 2017, 11:46am

multiple instances calling the same files is probably what generated your errors.
there is no gracefull way to close as far as i know.

i also use ctrl c.

but i only manually invoke appdaemon.
if you have it running automaticly you should never run it manually. (unless you are sure you closed everything and start it automaticly again after you closed the manual start)

SupahNoob · October 5, 2017, 3:35pm

What does utils.py look like? Running more than one instance of AppDaemon simultaneously should not cause the error you’re seeing in your service logs. My guess is that there is an error within utils that is not being recorded. If utils fails, then bedtimelights would not be able to resolve and would cause the error.

If you’ve only ever run AppDaemon manually, CTRL + C works just fine. If you need to stop the service, something like the below command should work:

sudo systemctl stop [email protected]

* @aimc I don’t remember, but it might be worth noting that handling SIGTERM and SIGKILL would be useful if we want AppDaemon to gracefully shut down when stopping the service. I’ve never had to play around with these and don’t remember if you already do this.