Export sensor data to csv. Create a new file automation after 2024.06 changes

I am a not so long-term HA user here (in the process of converting from Domoticz). I export my heat pump usage to csv file together with some environmental numbers.

The 2024.06 release was a bummer. The export did not work anymore in the end. I messed it up as I did not understand how the new way worked and in the days right after 2024.06 not much info was available. Decided to build the export automation from scratch using the new method and document it here for future use.

In the example below I use a more generic example of exporting weather data instead of heat pump.

Allowed external directory
Make sure HA is allowed to write to the folder where you want to store you exported files. Needed after 2024.06. In the example I will export the csv to /config/exportcsv folder.

  • Add the following to the configuration.yaml:
homeassistant:
  allowlist_external_dirs:
    - "/config/exportcsv"
  • Having - “/config” is OK too if you want to give a wider authorization.

Create a new entry for the file template using the new File integration

  • Settings > Devices & Services.
  • Click on the File integration to get to this screen:

  • There might or might not already some entities listed from the previous migration. Here it is about creating a new one. The cleaning of old/wrong entries is up to you :blush:.
  • Click ADD ENTRY
  • Choose “Set up a notification service”.
  • Enter the file path, i.e. /config/exportcsv/weather.csv
  • I chose not to have a time stamp as I create a timestamp myself within with the message I put the csv.
  • Click on submit.
  • You now have a new entry named (probably) “File [/config/exportcsv/weather.csv]”.
  • Rename this entry if you wish.
  • When renaming click on the 3dot menu > Rename. New name i.e.: Weather2File.[/config/exportcsv/weather.csv]
  • Now click on the “1 entity” text below the new entry.
  • The entity ID has unfortunately a generic name, in my case notify.file not related to purpose of the export. Rename this to keep an overview.
  • Click on the Entity ID > click on the gear icon.
  • Change the properties to for example, Name: “Export weather to csv”, Entity ID: notify.weatherfile.

Create the automation

  • In this example the automation runs time based, i.e. every x minutes.
  • Go to Settings > Automations & Scenes, click on the CREATE AUTOMATION button.
  • Select “Create new automation” (from scratch).
  • Open the ADD TRIGGER in the When section.
  • Select Time and Location (or anything else you want to use as a trigger).
  • Select Time Pattern.
  • When testing I use the minutes field, so it runs frequently. Minutes: /2 Runs every 2 minutes. Adjust it later to the timing you really need.
  • Open ADD ACTION in the Then do section.
  • Select: “Call Service”.
  • Search and select service: “notify.send_message”
  • Paste your message in the message field. If you copy / paste it from somewhere else check the syntax again. Pasting does change the formatting and, in my case, not for good. Maybe to correct my older syntax but it became less readable and some lines were now double in the code.
  • In the example with the Dutch buienradar weather service:
service: notify.send_message
data:
  entity_id: notify.weatherfile
  message: >-
    {{ now().strftime('%Y-%m-%d') , now().strftime('%H:%M:%S') ,
    states('sensor.wind_direction') ,
    states('sensor.wind_speed') ,
    states('sensor.wind_direction_azimuth') ,
    states('sensor.wind_gust') ,
    states('sensor.temperature') ,
    states('sensor.feel_temperature') ,
    states('sensor.precipitation_intensity') ,
    states('sensor.humidity') , states('sensor.visibility') ,         
    states('sensor.condition_code') ,
    states('sensor.detailed_condition') ,
    states('sensor.irradiance') ,
    states('sensor.wind_force') }}
  • I had to manually add the line with entity_id: notify.weatherfile. I copied from my Test-HA server and had a slightly different entity_id and the paste removes non-existing entities (I think).
  • Test your version of the message part first in the Developer Tools > Template menu!
  • Click on SAVE.
  • Give your new automation a name and description, for example: Export weather data to csv.
  • Click SAVE.

Troubleshooting

  • Check if the csv file is created after the trigger. In my case I got an error message. I had some typo’s in the message template.
  • In the automation windows there is a TRACES button where you can see what happened after the trigger in more detail.
  • Check Logbook and home-assistant.log.

Extra, daily rename of csv
I have another automation at 23:58 which starts a shell command to rename the csv to yyyymmdd – weather.csv
I don’t know how to do it from the HA GUI. This worked for me but maybe not the most elegant solution:

#!/bin/bash
current_date=$(date +'%Y-%m-%d')
original_filename="./exportcsv/weather.csv.csv"
new_filename="./exportcsv/weather.csv_${current_date}.csv"
mv "$original_filename" "$new_filename"

What I do not understand / to be improved

  • When 2024.06 was installed, it proposed to migrate my existing export. Did that and a message appeared to manually remove the old code from automations.yaml. Maybe it is in the wording but the old code was removed from automations.yaml and replaced with the new version of it. Don’t remove that! This new GUI method still uses the automations.yaml as place to store the automation. Don’t know if I did something wrong or that the place will change in a future release.
  • The csv created is not perfectly formatted. I need to improve my template formatting skills! I could not get rid of the round brackets without breaking other things and decimal comma instead if decimal point would be better for me. I use my OS/browser in US English but would like to have numbers formatted in my locale.
2 Likes

I can`t implement daily rename of csv. Can you show your automation action?

Did you read the shell command docs? Shell Command - Home Assistant

I suggest you do, as it will become apparent.

can you suppress the two header lines in the created csv file ?

#cat test.csv
Home Assistant notifications (Log started: 2024-10-11T08:31:34.362520+00:00)
--------------------------------------------------------------------------------
test,1234345
test,1234345