Rename files using regex?

Hi all,

My IP camera stores recordings/snapshots with the following naming convention:
/share/ipc/Entrance/2020-09-07/001/dav/16/16.11.19-16.11.19[M][0@0][0].mp4_
/share/ipc/Entrance/2020-09-07/001/jpg/16/11/22[M][0@0][0].jpg

I’d like to rename them (upon catching an event via folder_watcher integration) as follows:
Entrance_2020-09-07_16_11_19.mp4
Entrance_2020-09-07_16_11_22.jpg

I’ve never been good at regex and I assume that the way to achieve that. Would really appreciate any tips on composing such a regex or solving it another way.

Thanks!

There is a Linux application called rename. (Simple guide and examples)

Re-read and your issue is that you want to rename a file based on folder and file names.

I have a feeling that the file names you used are not actual file names, but your attempt at Regex. It would be a lot more helpful if you provided actual examples; just like you did for the expected results!

You asked for regex, but I ended up using a bunch of awk and parameter expansion to get what you want.

That way, it’s easier for you to understand as I am dealing with delimiters.

Anyways, I wrote you this:

#!/bin/bash
  
# /share/ipc/Entrance/2020-09-07/001/dav/16/16.11.19-16.11.19[M][0@0][0].mp4
# Entrance_2020-09-07_16_11_19.mp4

# Source path
src="/share/ipc/"
# Destination path
dst="/change/me/please"

for file in $( find "$src" -name "*.mp4" ); do

        location="$(echo $file | awk -F'/' '{print $4}')"
        ymd="$(echo $file | awk -F'/' '{print $5}' | tr '-' '_' )"
        file_name="$(echo $file | awk -F'/' '{print $NF}' | tr '.' '_' )"
        time="${file_name%-*}"
        renamed="$location"'_'"$ymd"'_'"$time"'.mp4'

        echo "Location: $location and Ymd: $ymd and time: $time"
        # Do not comment this out until you _know_ that the outputs are correct
        # mv "$src/$file" "$dst/$renamed"
        echo "Test instruction: $file" "$dst/$renamed"

done         

This version of the script is safe, as I have commented out the mv operation and all it does is echo so you can see what it is doing.

DO NOT UNCOMMENT THE MOVE COMMAND WITHOUT TESTING

Bad things happen when you don’t understand or at least test with some dummy files!

Of course, change the file to suit your needs and reply with any questions you might have.

Thanks for looking into this fanuch, really appreciate your help! Your input has put me on the right track and I’m mostly done with the script - see the outstanding issue below.

To recap, I have a IP camera that akes snapshots and records video clips upon detecting motion and stores them on an FTP server. The default file paths are not so user-friendly and my goal is to improve them before uploading them to Google Drive (see the samples below).

The starting point of my automation is the folder_watcher component with the following config watching for the snapshots/videos to be posted an FTP share by the camera:

folder_watcher:
  - folder: /share/ipc/
    patterns:
      - '*.jpg'
      - '*.mp4'

Then it’s getting picked up by the following automation:

- id: '1599477490330'
  alias: Outdoor Cam Rec GDrive Upload upon Motion
  description: ''
  trigger:
  - event_type: folder_watcher
    platform: event
  condition:
  - condition: or
    conditions:
    - condition: template
      value_template: '{{ ''moved'' in trigger.event.data.event_type }}'
    - condition: template
      value_template: '{{ ''created'' in trigger.event.data.event_type }}'
  - condition: or
    conditions:
    - condition: template
      value_template: '{{ ''.jpg'' in trigger.event.data.file }}'
    - condition: template
      value_template: '{{ ''.mp4'' in trigger.event.data.file }}'
  action:
# Added a delay in case the issue was related to the file being locked by the FTP service upon storage and attempting to run mv too fast, didn't help  
  - delay: 00:00:01
# Counting the number of motion detection events in the UI for troubleshooting
  - data: {}
    entity_id: input_number.entrance_motion_counter
    service: input_number.increment
# Running the bash script to improve the naming of files
  - data_template:
      path: '{{ trigger.event.data.path }}'
    service: shell_command.ipc_gdrive_prep
# Logging the path to .jpg/.mp4 reporting by folder_watcher to troubleshoot things  
 - data_template:
      message: '{{ trigger.event.data.path }}'
    service: system_log.write
  mode: queued
  max: 10

shell_command.ipc_gdrive_prep is called the following way:

ipc_gdrive_prep: bash /config/ipc_gdrive_prep.sh {{ path }}

and consists of:

#!/bin/bash
  
# Expected $1 input in case of .mp4_ (for some reason the camera stores with this file extension first and folder watcher neven catches the moment of changing to the proper file extension, so we'll have to strip the underscore off below):
# /share/ipc/Entrance/2020-09-07/001/dav/17/17.26.58-17.27.42[M][0@0][0].mp4_
# Expected output in case of .mp4_:
# Entrance_2020-09-07_16_11_19.mp4

# Expected $1 input in case of .jpg:
# /share/ipc/Entrance/2020-09-07/001/jpg/16/11/22[M][0@0][0].jpg
# Expected output in case of .jpg:
# Entrance_2020-09-07_16_11_22.jpg

dst="/share/ipc_gdrive_prep"
file_ext="$(echo ${1%%_} | awk -F'.' '{print $NF}')"
camera_name="$(echo ${1%%_} | awk -F'/' '{print $4}')"
date="$(echo ${1%%_} | awk -F'/' '{print $5}' )"

if [ "$file_ext" == "mp4" ]; then
    file_name="$(echo ${1%%_} | awk -F'/' '{print $NF}' | tr '.' '_' )"
    time="${file_name%-*}"
    renamed="$camera_name"'_'"$date"'_'"$time"'.'"$file_ext"

    mv -f "${1%%_}" "$dst/$renamed"
    #echo "Test instruction: ${1%%_}" "$dst/$renamed"
elif [ "$file_ext" == "jpg" ]; then
    time="$(echo ${1%%_} | awk -F'/' '{print $8}' )_$(echo ${1%%_} | awk -F'/' '{print $9}' )_$(echo ${1%%_} | awk -F'/' '{print $10}' | cut -c1-2)"
    renamed="$camera_name"'_'"$date"'_'"$time"'.'"$file_ext"

    mv -f "${1%%_}" "$dst/$renamed"
    #echo "Test instruction: ${1%%_}" "$dst/$renamed"
else
    exit 42
fi

The camera always stores the .jpg first and then .mp4. This script works perfectly fine with SSH, but not within the above mentioned HA automation.

The first occurence of the automation (.jpg) works just fine. However, when it runs the second time (.mp4), I get “Error running command: bash /config/ipc_gdrive_prep.sh {{ path }}, return code: 1” in the logs. Tried editing the “mp4” branch of if statement in above mentioned script to match the “jpg” condition, but that hasn’t solved the issue.

Any thoughts on how to trouleshoot this further?

Finally sorted this out.

Turns out the camera renames the mp4 file right after the recording, which is not getting noticed by the Folder watcher component:

Path captured by Folder watcher:
/share/ipc/Entrance/2020-09-14/001/dav/20/20.23.19-20.23.19[M][0@0][0].mp4_

Actual file available in the folder (notice the change in the second timestamp of the filename; removal of the underscore from the file extension is neglectable):
/share/ipc/Entrance/2020-09-14/001/dav/20/20.23.19-20.23.54[M][0@0][0].mp4

The following adjustment to the mp4 part of the script from my previous message will get things working:

if [ "$file_ext" == "mp4" ]; then
    file_name="$(echo $1 | awk -F'/' '{print $NF}' | tr '.' '_' )"
    time="${file_name%-*}"
    renamed="$camera_name"'_'"$date"'_'"$time"'.'"$file_ext"
# The camera manages to rename the .mp4 file (without Folder watcher capturing that), so $1 is no longer valid.
# We need to detect the valid path to .mp4 based on the outdated $1 we know:
    detectpath="$(find $(dirname $1) -name $(echo $time | tr '_' '.')*.mp4 )"

    mv -f "$detectpath" "$dst/$renamed"
#    echo "Test instruction: $detectpath" "$dst/$renamed"

This works with the FTP upload feature of my IMOU Dome Lite camera and should probably hold valid for any IMOU / Dahua camera supporting web interface and/or CGI commands