Logging data to one file per day

Configuration
1

I’ve been reading a bit, trying to figure out what the best approach is, but I’m not really sure yet.

My end goal is to log a ton of data, from primarily MQTT sources, and have it automatically upload to my Google Drive. I’d like the data split into files each containing just a single day. Ideally the files should be plain text, as this is easily imported in to most programs.

Currently I have InfluxDB installed, but I can’t find where the data is stored.

TL:DR: I want to save MQTT data to Google Drive, each day a separate file, in plain text format.

2

Simplest way: Use the file notification, a shell script and create a cron job.

1 Like
3

Reading the documentation I can’t see if it possible to aggregate multiple “notifications” into a single file. I.e. all measurements from a Sonoff POW into a single file.

Do you know if that’s possible?

4

Just gonna reply to myself here… found this that sounds promising: Export sensor data to text file

22:39 update:

So far so good! Now getting data logged to a .csv file. However, I’ve specified:

trigger:
  - platform: time
    seconds: 10

Yet the data is only written once a minute:

Home Assistant notifications (Log started: 2018-06-30T20:34:10.974069+00:00)
--------------------------------------------------------------------------------
2018-06-30T20:34:10.974233+00:00 , 284, 229
2018-06-30T20:35:10.944721+00:00 , 281, 230
2018-06-30T20:36:10.940931+00:00 , 318, 229
2018-06-30T20:37:10.943042+00:00 , 281, 229

Howcome? Data is being sent from the sensor via MQTT once every 10 seconds.

(Bonus question: Is it possible to add headers for the data an easy way? I guess I could just run an automation that creates the file with the headers…)

5

It’s more: all or nothing.

To have more options exporting the needed data from the database once a day would be another option. Check this blog post for details.

6

That’s definitely a possibility as well. With the current “save value every 10 second” solution, I guess there’s a risk of saving duplicate values if a value hasn’t been updated. I feel that isn’t the proper approach.

I’d like all of the data management to be contained within HA, so the method covered in “Visualize your IoT data” is not optimal.

7

Some platforms support force_update and if it is not set it shouldn’t happen. Still depends on the platform.

You were talking about exporting the data from Home Assistant and exporting the data out of the DB will give you the most flexibility.

8

From the link that you provided, it seems like that method is manual. Or is there a way to make it automatic? I only have a limited experience with HA, and is still learning.

My current code works almost as it should. For some odd reason it logs every minute, not every 10 seconds.

  - alias: sensor_values_to_file
    initial_state: 'on'
    trigger:
      - platform: time
        seconds: 10
    action:
      - service: notify.filenotify
        data_template: 
          message: '{{now().year}}-{{now().month}}-{{now().day}} {{now().hour}}:{{now().minute}}:{{now().second}}:{{now().microsecond}}, {{ states.sensor.power.state}}, {{states.sensor.voltage.state}}'
9

A bash script and a cronjob.

10

If it all possible, I’d like to keep all the code within HA. I had it setup with this:

notify:
  - platform: file
    name: mqtttofile
    filename: {{ '/share/ '{{now().year}}-{{now().month}}-{{now().day}}'.csv
    timestamp: false

However it just creates a file named ‘{{now().year}}-{{now().month}}-{{now().day}}’.csv. The idea was that whenever the date changed, the filename would also change and a new file would automatically be made. Neat… if it worked.

Someone suggested doing it with a shell_command instead, and posted this:

shell_command:
  write_data_to_file: "echo 'data' | tee -a /share/{{now().strftime('%Y-%m-%d.csv')}}"

Which is called with:

  - alias: mqtt_to_file
    initial_state: 'on'
    trigger:
      - platform: mqtt
        topic: "tele/sonoff01/SENSOR"
    action:
      - service: shell_command.write_data_to_file
        data_template: 
          message: '{{ trigger.payload }}'

But that just results in:
ERROR (MainThread) [homeassistant.components.shell_command] Error running command: echo ‘message’ | tee -a /share/{{now().strftime(’%Y-%m-%d.csv’)}}, return code: -11

I’m open to any ideas. I don’t have much experience with HA, so I might miss otherwise obvious mistakes.

11

I want to do the same. Could you find a solution for that?

12

A trigger every x minutes that uses notify to a file. You’ll need to add the file to your config file.
I can provide the code required?