Folder watcher (Watchdog) component

I did try this approach-- you are correct in that it does work, but it will not function correctly if there are files which are detected in a sub-folder within the movie directory. I need to send notifications for all files in the /media/movie folder recursively. Your check will result in notifications for any file in /media/movie, but not a file in /media/movie/sub_folder.

It’s with this limitation in mind that I’m trying to search trigger.event.data.folder for any sub-string matching /media/movie- so it would catch files inside sub-folders as long as /media/movies is somewhere in the file path. But something is not working properly and I have no yet figured out what.

I’m new in the forums here and realizing this may not be the best place to be asking about this. I may make a new post where appropriate for further discussion. If so, I’ll update this post with a link.

I’ve not been following the conversation but it appears it might be simpler just to manually code multiple automations, rather than using templating to handle all the different cases

Robin - I’m somewhat new to working with Home Assistant automation and did not realize that was possible. I know you can easily create multiple automations, but how do I associate a given folder watcher automation with a particular folder I’d like folder watcher to monitor?

For sake of clarity, I’ll list an example. I have folder watcher configured as follows in my configuration.yaml:

#Folder Watcher Setup
folder_watcher:
  - folder: /media/movies
  - folder: /media/tv

I know I can easily create multiple automations based on a file being either created, deleted, modified, or moved in one of those two directories (or any of their sub-directories). But how can I associate a particular folder (or set of folders) with a particular automation? I think I almost need to operate multiple instances of folder_watcher, or have folder_watcher split itself into two entities which are configured differently. Does that make sense? I feel like I’m missing something basic here

I would add the check if '/media/movie' in {{ trigger.event.data.folder }} as a condition for the automation. This will simplify your yaml later

Ah, okay that makes sense. Multiple automations with the different use cases handled by conditions, rather than a bunch of logic in the template to handle them, does make things a bit simpler.

I’ve written the following two automations based on your advice here:

#Send discord notification when new movie is added.
- alias: New file alert - Movies
  trigger:
    platform: event
    event_type: folder_watcher
    event_data:
      event_type: created
  condition:
    condition: template
    value_template: "{{ if '/media/movies' in {{ trigger.event.data.folder }} }}"
  action:
    service: notify.discord
    data_template:
      title: New media uploaded.
      target: "888888888888888888"
      message: "__**Movie Added:**__ {{ trigger.event.data.file }} in {{ trigger.event.data.folder }}"

#Send discord notification when new tv episode is added.
- alias: New file alert - TV
  trigger:
    platform: event
    event_type: folder_watcher
    event_data:
      event_type: created
  condition:
    condition: template
    value_template: "{{ if '/media/tv' in {{ trigger.event.data.folder }} }}"
  action:
    service: notify.discord
    data_template:
      title: New media uploaded.
      target: "888888888888888888"
      message: "__**TV Episode Added:**__ {{ trigger.event.data.file }} in {{ trigger.event.data.folder }}"

Unfortunately, I keep getting the following error from Home Assistant about my automation.

Invalid config for [automation]: invalid template (TemplateSyntaxError: expected token ‘end of print statement’, got ‘string’) for dictionary value @ data[‘condition’][0][‘value_template’]. Got None

Any idea what my problem could be? I’ve been researching the error, and thought it might be an issue with single vs. double quotes in my automation, but I think everything is correct. Kind of at a loss again.

be careful when using special characters like * and also check your single and double quote usage

I double checked my quote usage, and eliminated my special character usage just to eliminate that possibility for failure. I also peared my automation down as much as possible, and included just one automation of the two I plan to eventually use. Still no dice. The automation I’m currently using is as follows:

#Send discord notification when new movie is added.
- alias: New file alert - Movies
  trigger:
    platform: event
    event_type: folder_watcher
    event_data:
      event_type: created
  condition:
    condition: template
    value_template: "{{ if '/media/movies' in {{ trigger.event.data.folder }} }}"
  action:
    service: notify.discord
    data_template:
      message: "Movie Added: {{ trigger.event.data.file }} in {{ trigger.event.data.folder }}"
      target: !secret discord_channel_id

When Home Assistant starts up, I get a notification in the gui saying:

The following components and platforms could not be set up: Automation. Please check your config.

The log then displays the following error:

ERROR (MainThread) [homeassistant.config] Invalid config for [automation]: invalid template (TemplateSyntaxError: expected token ‘end of print statement’, got ‘string’) not a valid value for dictionary value @ data[‘condition’][0][‘condition’]. Got None

Eliminating the condition from the automation results in the automation functioning just fine, so I’m pretty sure the issue is somewhere in these three lines:

  condition:
    condition: template
    value_template: "{{ if '/media/movies' in {{ trigger.event.data.folder }} }}"

but I’m not sure what is causing it.


Edit: I’ve since tried a couple other things. I went back to the approach of including the if 'media/movie in {{ trigger.event.data.folder }} line inside some if/else logic in my template. I tried including the logic as both a single line, and using multiline string. Both approaches yield a similar error to the oneI got earlier when including the check as a condition.

The two different versions of the automation I tried are as follows:

#Send discord notification when new movie is added.
- alias: New file alert - Movies
  trigger:
    platform: event
    event_type: folder_watcher
    event_data:
      event_type: created
  action:
    service: notify.discord
    data_template:
      message: "{% if '/uploads/MKV' in {{ trigger.event.data.folder }} %} Movie Added {{ trigger.event.data.file }} in {{ trigger.event.data.folder }} {% else %} File added to non-relevent folder {% endif %}"
      target: !secret discord_channel_id
#Send discord notification when new movie is added.
- alias: New file alert - Movies
  trigger:
    platform: event
    event_type: folder_watcher
    event_data:
      event_type: created
  action:
    service: notify.discord
    data_template:
      message: >
        {% if "/uploads/MKV" in {{ trigger.event.data.folder }} %}
          Movie Added {{ trigger.event.data.file }} in {{ trigger.event.data.folder }}
        {% else %}
          File added to non-relevent folder 
        {% endif %}
      target: !secret discord_channel_id

Both result in the same error:

ERROR (MainThread) [homeassistant.config] Invalid config for [automation]: invalid template (TemplateSyntaxError: expected token ‘:’, got ‘}’) for dictionary value @ data[‘action’][0][‘data_template’][‘message’]. Got None.


Edit 2:
After further screwing around: I think all of these errors have to do with using the variable {{ trigger.event.data.folder }} within the template logic? All three of these approaches work fine in the Home Assistant Template Editor if I manually declare a string variable with some file path as its value at the beginning. For some reason, the template is not pulling the value of {{ trigger.event.data.folder }} correctly, unless it is as the contents of the message being sent?


Edit 3:
So sorry for what has become a wall of text, but I think I’m about to the bottom of this. I can say with almost certainty that the problem is caused by not correctly calling the {{ trigger.event.data.folder }} variable within the logic of the template. The automation works perfectly if you manually set a variable name, as I did in the code below. But it produces an error if you use the variable {{ trigger.event.data.folder }}. I suspect the same is true when referencing this variable in a condition? It’s possible I’ve just wasted the better part of a day and have no idea what I’m talking about, but I think I’ve about narrowed it down here.

#Send discord notification when new movie is added.
- alias: New file alert - Movies
  trigger:
    platform: event
    event_type: folder_watcher
    event_data:
      event_type: created
  action:
    service: notify.discord
    data_template:
      message: >
        {% set path = '/uploads/MKV/movies/movie1' %}
        {% if '/uploads/MKV' in path %}
           Movie Added {{ trigger.event.data.file }} in {{ trigger.event.data.folder }}
        {% else %}
           File added to non-relevent folder 
        {% endif %}
      target: !secret discord_channel_id

Personally I do not use complicated yaml but instead write python_scripts, might be worth a try

1 Like

@juan11perez @robmarkcole So I finally got it figured out. The problem in all these different approaches was wrapping the trigger.event.data.folder variable in double curly brackets, like this:

{{ trigger.event.data.folder }}

Wrapping the variable name in brackets is necessary when including the contents of the variable as part of the message to any of the notify integration platforms ( in my case, discord) , but it is not required when referencing the variable as part of the template logic.

From the beginning, the goal of all my automation has been to send a differently formatted discord message depending on which folder folder_watcher detects a file in. I attempted to do this with some if/else logic on the string contents of the trigger.event.data.folder variable provided by folder_watcher, exemplified in the following line:

{% if “/media/movies” in {{ trigger.event.data.folder }} %}

I also tried including this check as a condition to the automation running, like this:

condition:
    condition: template
    value_template: "{{ if '/media/movies' in {{ trigger.event.data.folder }} }}"

In both cases, I was met with an error similar to this one:

Invalid config for [automation]: invalid template (TemplateSyntaxError: expected token ‘:’, got ‘}’)

It was reading this post from @albert1 that finally made it start to click for me. I’m not by any means an expert on Home Assistants architecture, but I believe the reason for this is that the double brackets around trigger.event.data.folder are only needed in the message contents portion of the code because this is the only portion where Home Assistant is not already rendering the template. In the messages content, HA needs to be explicitly told to get the contents of the variable from the template, thus the double brackets, but, in the template logic, the {% %} around the whole statement have already got Home Assistant rendering a template, and thus it recognizes the trigger.event.data.folder variable without having to flag it specifically.

The same should hold true for preforming the check in the conditions, as Rob recommended. This will not work:

condition:
    condition: template
    value_template: "{{ if '/media/movies' in {{ trigger.event.data.folder }} }}"

but this should:

condition:
    condition: template
    value_template: "{{ '/media/movies' in trigger.event.data.folder }}"

(Notice above that, in addition to dropping the double curly brackets around trigger.event.data.folder, I also dropped the “if” check entirely, leaving just in. That is because condition templates only check for whether or not the given condition renders true- basically, the logic of “if this is true, continue, otherwise dont do anything” is already taken care of for us.)

TL;DR: Only wrap the trigger.event.data.folder variable in double curly brackets when including it as part of the message output. The rest of the time, HA is already rendering the template and knows what the variable is without the {{ }} flag. My complete, and working (!!) automation is as follows:

- alias: 'Send notification to Discord when new media is added to upload directory'
  trigger:
    platform: event
    event_type: folder_watcher
    event_data:
      event_type: created
  action:
    service: notify.discord
    data_template:
      message: >
        {% if "/uploads/MKV" in trigger.event.data.folder %}
          Movie Added file {{ trigger.event.data.file }} folder {{ trigger.event.data.folder }}
        {% elif "/uploads/TV_Seasons" in trigger.event.data.folder %}
          TV Episode Added file {{ trigger.event.data.file }} folder {{ trigger.event.data.folder }}
        {% else %}
          Error, folder path not configured for notification
        {% endif %}
      target: !secret discord_channel_id

I’m still considering whether or not to split this into multiple automatons using automation conditions, as @robmarkcole recommended, but right now I’m leaning towards leaving the logic in the template, as this automation serves one purpose, I just want it to send slightly different messages depending on the folder in which the file is detected.

Being able to reliably preform different actions depending on the directory where a folder watcher event is detected greatly increases the utility of folder watcher for me- Hope this can help someone else

1 Like

thank you for sharing your findings. It’s all useful.

Hi!

I the component only able to detect changes made to the folder locally? I’ve got CIFS shares mounted to /media/xxx. When I touch a file within that directory locally (on the machine where HA runs), it detects the creation of the file. If I create / move / modify a file from the remote location, it doesnt. Is the any way this or any other component can react to changes made remotely?

Thanks!

Hi there,

i have been trying to have the watchdog working for sometimes now without too much success.

I’m trying to monitor a folder where my external cam is saving the pictures it takes once movement is detected.
One thing particular with my cam is it creates a different folder structure per day / hours ie: 20200103/10 , 20200103/11 , etc . You see the drill :slight_smile:

I’m monitoring the root folder for that cam ( which is homeassistant/www/camext/“folder for date and time here”
there is something fishy, as if i create a file manually it triggers both the node-red file watch node and the folder watcher , but none of the files that are saved by my cam in the same folder is triggering either.

Here’s my config just in case:

  whitelist_external_dirs:
    - /config/www/camext
    - /config/www/camext/Picture


folder_watcher:
  - folder: /config/www/camext/Picture
    patterns:
      - '*'


  alias: New file alert
  trigger:
    platform: event
    event_type: folder_watcher
    event_data:
      event_type: created
  action:
    service: notify.notify
    data_template:
      title: New image captured!
      message: "Created {{ trigger.event.data.file }} in {{ trigger.event.data.folder }}"
      data:
        file: "{{ trigger.event.data.path }}"
cat test1.txt
total 17408
drwxrwxrwx 2 root root      0 Jan  3 16:11 ..
-rwxrwxrwx 1 root root 145155 Jan  3 16:11 CAMEXT1578085876_0686.jpg
-rwxrwxrwx 1 root root 145455 Jan  3 16:11 CAMEXT1578085876_2304.jpg
-rwxrwxrwx 1 root root 145601 Jan  3 16:11 CAMEXT1578085876_3686.jpg
-rwxrwxrwx 1 root root 136514 Jan  3 16:11 CAMEXT1578085876_5232.jpg
-rwxrwxrwx 1 root root 144818 Jan  3 16:11 CAMEXT1578085921_0422.jpg
-rwxrwxrwx 1 root root 144814 Jan  3 16:11 CAMEXT1578085921_2050.jpg
-rwxrwxrwx 1 root root 144025 Jan  3 16:11 CAMEXT1578085921_3684.jpg
-rwxrwxrwx 1 root root 143814 Jan  3 16:11 CAMEXT1578085921_5858.jpg
-rwxrwxrwx 1 root root 145136 Jan  3 16:18 CAMEXT1578086295_0344.jpg
-rwxrwxrwx 1 root root 145077 Jan  3 16:18 CAMEXT1578086295_3478.jpg
-rwxrwxrwx 1 root root 144581 Jan  3 16:18 CAMEXT1578086295_5476.jpg
-rwxrwxrwx 1 root root 143143 Jan  3 16:18 CAMEXT1578086296_1942.jpg
-rwxrwxrwx 1 root root   1037 Jan  3 16:24 test1.jpg
-rwxrwxrwx 1 root root 140736 Jan  3 16:29 CAMEXT1578086978_0600.jpg
-rwxrwxrwx 1 root root 140597 Jan  3 16:29 CAMEXT1578086978_1954.jpg
drwxrwxrwx 2 root root      0 Jan  3 16:29 .
-rwxrwxrwx 1 root root 138951 Jan  3 16:29 CAMEXT1578086978_3566.jpg
-rwxrwxrwx 1 root root 139199 Jan  3 16:29 CAMEXT1578086978_5284.jpg
-rwxrwxrwx 1 root root      0 Jan  3 16:43 test1.txt

as far as i know it should be working , please let me know if someone sees something obvious :slight_smile:

thanks guys !

You could try dropping this condition

thanks rob but no luck, already tried it for modified as well and no condition.

As mentionned above, when i create my tests files they get detected but when the system creates them they are not . i looked into a possible permission issue login in the hassio docker on the command line to see if home assistant have access to the files, and i can browse and access the files. I can as well access the pictures from the home assistant URL /local and specifying the path and file name.

Honestly , i’m really at lost here :frowning:

if any of you guys have a suggestion, i’ll be happy to try them :slight_smile:
thanks

Oh if you are using docker that could be causing issues. In particular file paths might not always be what you expect. Also the whitelisting by HA could be compounding the issue. You could write a quick script using watchdog to see what events are being fired and check the path in them

Say I want to use this to detect an image captured by my camera’s motion detection and to send the picture via Telegram to my phone. I then want to use a shell command to delete the file.

There’s a lag when the file uploads to Telegram. What if during this time folder watcher fires the automation again, because the camera’s motion detection is tripped again? I get that the first repetition of the automation will have to finish before the second starts, but will the data from the folder watcher have changed mid-first automation? I.e. will my shell command delete the new file, not the one it’s just sent?

How do I mitigate this?!

Thanks!

Just posted in another thread but think I probably should have posted in here. I just got folder watcher working to send me a copy of an uploaded image via telegram. It works, but as soon as the image appears the automation fires repeatedly, sending me hundreds of the same image. It’s so rapid it renders my whole setup completely unresponsive! Has anyone seen anything like this? I’ve had no joy finding anything similar on here.

Thanks!

Well the component has been live for a while yet so its pretty battle hardened. Can you provide more info - OS, what is creating the files, any logs?

Not really up on the logs… Can you let me know exactly what you need? I’m on HassOS, tried doing files in both via samba and FTP uploading via my hikvision cam.

Thanks!

OK some more detail.

YAML (config):

folder_watcher:
  - folder: /share/camfile
    patterns: '*.jpg'

YAML (automation):

- id: '1581084936624'
  alias: Camera Push Notifications
  description: ''
  trigger:
  - event_data:
      event_data:
        event_type: created
    event_type: folder_watcher
    platform: event
  condition: []
  action:
  - data_template:
      data:
        photo:
        - caption: Motion detected by the CCTV system.
          file: '{{ trigger.event.data.path }}'
      message: Motion detected by the CCTV system.
    service: notify.telegram_joey

I just tested again and get a minimum five copies of the same image. Logs blank, should I be setting a higher level in the logs or something?

Thanks.

EDIT: config also has folder whitelisted.