Folder Watcher | file sensor?

Configuration
#1

trying to use event data to get file name then file sensor to pull data i want from that file but no luck

event_type: folder_watcher
data:
  event_type: modified
  path: /config/BlueLog/2023-01-20 112500.csv
  file: 2023-01-20 112500.csv
  folder: /config/BlueLog
origin: LOCAL
time_fired: "2023-01-20T16:26:00.134125+00:00"
context:
  id: 01GQ800MJ6XC9TY393P5VS1ZF1
  parent_id: null
  user_id: null

input_text:

  last_added_file:
    name: Last added file
    initial: None

i receive file name using input text method being called by automations using

alias: fwa
description: ""
trigger:
  - platform: event
    event_type: folder_watcher
    event_data:
      event_type: created
    id: created
  - platform: event
    event_type: folder_watcher
    event_data:
      event_type: modified
    id: modified
condition: []
action:
  - choose:
      - conditions:
          - condition: trigger
            id: modified
        sequence:
          - service: input_text.set_value
            data:
              value: " {{ trigger.event.data.path }} "
            target:
              entity_id: input_text.last_added_file
mode: single

my idea was to then use that input_text for file path but it doesnt want to work

  - platform: file
    name: Water Temp
    file_path: 'input_text.last_added_file'
    value_template: '{{ value.split(",")[5] }}'
    unit_of_measurement: F
#2

Right that doesn’t work, hard-coded file path only.

A command line sensor accepts templates in the command. So you could read all or part of the file into a sensor that way after sticking the name in an input text.

Out of curiosity, what are you trying to do though? Where do these files come from and why does the source of data for the sensor keep changing?

1 Like
#3

"python3 -c “print(open(‘input_text.last_added_file’).readlines())”

this works to get other files but not random… any help?

sensor:
  - platform: command_line
    name: 'waPh'
    command: "python3 -c "print(open('input_text.last_added_file').readlines())"
    unit_of_measurement: "pH"
    value_template: '{{ value.split(",")[4] }}'

havent tailed it yet either just trying to make it work

#4

The text input_text.last_added_file is just that text. HA does not replace all strings that happen to look like an entity ID with their state in random things, you have to actually use a template.

python3 -c "print(open('{{ states("input_text.last_added_file") }}').readlines())
1 Like
#5

this works great thank you! whats easiest way to tail this? i had to add a few \ in

sensor:
  - platform: command_line
    name: 'waPh'
    command: "python3 -c \"print(open('{{ states(\"input_text.last_added_file\") }}').readlines())\""
    unit_of_measurement: "pH"
    value_template: '{{ value.split(",")[4] }}'
#6

I’m not aware of any way to tail a file on disk and get that data into HA that way. The command line and file integrations are both polling. You can speed up that polling by using homeassistant.update_entity but that’s about it.

Again what are you actually trying to do? Where are these files coming from and what is updating them? There might be a better way to do this but I don’t have all the information.

#7

coming from my bluelab connect data being save to home assistant

i used to have node red running a crazy auto dosing program until the recent update crippled it… i didnt see it in time to use a backup so here i am no node red to scrap data from one csv file to the next and im trying to work around the issue

so idea was use same method on home assistant itself

bluelabs wont allow saving to direct file name for whatever reason, so that changes but in same folder.

i need that data to display into home assistant then i can use those values to control doser in automations

#8

Hm. So if the only way HA can see that data is files you’re going to have a difficult time I think. Like I said there’s really no way to live tail a file into HA and I don’t see anything in HACS either. Which leaves you with a few options:

  1. Write your own custom integration that tails a file and updates the state of a sensor whenever it changes. I’m not aware of any particular reason such an integration couldn’t exist I just don’t know any that do that. It’s also possible there is a custom integration for that and its just not in HACS or Core currently, might find something searching around the forum or Google.
  2. Modify the external service to fire events at HA instead of writing out files via the REST API
  3. If #2 is impossible, create an intermediary service which tails the file and fires events at HA via the REST API. This service could be an add-on if you want it to run alongside HA or just host it somewhere else.
  4. Use the Node RED or AppDaemon addons to tail the file and send events to HA when it updates. This is really just a variation of #3, NR or AD are acting as the intermediary services in this case. If you find it easier to use them then writing something yourself. pyscript might also work here even though its technically an integration rather then a separate service, not sure.

Basically if you can’t make/find #1 then you want the data to come to HA in the form of events rather then files. If you have events then you can make automations from those events or turn that data into a live-updating sensor using a trigger template sensor. But there’s not really a good way to handle live updating files OOTB.

#9

is it possible to have it save to a file? if i could tell it to save to a file of my choice i could then use the file sensor to pull data

myfile.csv

#10

Sorry tell what to save to a file? And it really depends on how often you’re writing data. The file integration is type local polling so its not live tailing anything. The sensor also simply shows you the last line in the file as it says.

So if you’re adding lines to the file occaisionally on a predictable schedule then sure that’s fine. If you’re going to be streaming data into it very quickly all or most of the time then that won’t work well.

#11

thats how it was being used before. it updates once a minute and would allow me to get data needed

it just happened to use node red to complete the process then i could use a file sensor and pull data from last line which is all i need final line…which would watch folder for changes then save that to file of my choosing.

  - platform: file
    name: Water Temp
    file_path: /config/BlueLog/2023-01-20 112500.csv
    value_template: '{{ value.split(",")[5] }}'
    unit_of_measurement: F

  - platform: file
    name: Water ph
    file_path: /config/BlueLog/2023-01-20 112500.csv
    value_template: '{{ value.split(",")[4] }}'
    unit_of_measurement: Ph

  - platform: file
    name: Water Ppm
    file_path: /config/BlueLog/2023-01-20 112500.csv
    value_template: '{{ value.split(",")[3] }}'
    unit_of_measurement: Ppm

  - platform: file
    name: Water Time
    file_path: /config/BlueLog/2023-01-20 112500.csv
    value_template:  '{{ value.split(",")[0] }}'
#12

i just cant find path anymore to be able to pull file…

or can get path not just end of file…

#13

I’m pretty lost tbh. I guess I’ll try and answer some questions you’ve had:

Yes. HA can write to a file with a file notifier or a command line notify.

ok polling is probably fine then.

If these are your file paths then its not going to be possible to use a file sensor since its got a date on it. A file sensor needs a hard-coded file path and that file path is clearly changing regularly. The command line sensor is a better bet.

I guess I’m confused why HA is looking at files at all if Node RED is involved. Node RED can simply fire events at HA. One event per data update with all the data in it. Then an automation listening for that event and processing it however you wish. Seems a lot easier then having NR write to a file and HA trying to read that back in via 4 different file sensors but that’s just me.

If you want it in sensors for example that’s easy too. Have NR fire an event at HA with data like this per data update:


{
  "temp": 123,
  "ph": 123,
  "ppm": 123,
  "time": "22231111T000000.000 GMT"
}

And then make trigger template sensors like this (we’ll say the event type is WATER_READING for this example:

template:
  - trigger:
      - platform: event
        event_type: WATER_READING
    sensor:
      - name: Water temp
        state: "{{ trigger.event.data.temp }}"
        device_class: temperature
        unit_of_measurement: °F
        state_class: measurement
      - name: Water ph
        state: "{{ trigger.event.data.ph }}"
        unit_of_measurement: Ph
        state_class: measurement
      - name: Water ppm
        state: "{{ trigger.event.data.ppm }}"
        unit_of_measurement: Ppm
        state_class: measurement
      - name: Water time
        state: "{{ trigger.event.data.time }}"
        device_class: timestamp
#14

mainly because i havent a clue what im doing its a cross fingers and hope it works method.

didnt know you could make sensors that way, i take the long way around because of my limitaions/knowledge of computers . half the stuff ive done is a frankenstein project of bits of de from various sources made to work together…

node red made things easy to do and i read on fourm someone using sensor file to do something similar so i used their code to get there

when i lost ability to use node red it really hurt the system. just trying to maintain at this point and i felt if i could get monitoring systems back online it would help with progress.

pumps are up and running, just gotta figure out how to make these things function… without node red …

#15

@CentralCommand got it with

sensor:

  • platform: command_line
    name: chss
    command: “tail -n 1 ‘{{ states(“input_text.last_added_file”) }}’ > /config/bluelab.csv”

i then use a file sensor to pull data…its already setup to do it so…

1 Like