I made a shell script to update blueprints…
I don’t use a lot of blueprints, but figured this would be easier then checking them manually from time to time.
It should probably send a notification or something when a blueprint is updated, but didn’t bother to add that yet.
if you have a script which doesn’t have a source_url in the source file, add it to your local copy with “custom-” in front of the url source_url: custom-https://example.com/source.yaml the script will then use that url, and re-insert the source url after downloading. edit: if the newly downloaded file has a source_url, the custom url isn’t inserted
Create the following script in the /config/scripts/ directory
/config/scripts/blueprints_update.sh
#!/bin/bash
DeBuG="false"
function _blueprint_update_debug
{
if [ "${DeBuG}" == "true" ]
then
echo "$@"
fi
}
cd /config/blueprints/
for file in ./*/*/*.yaml
do
echo "> ${file}"
# get source url from file
blueprint_source_url=$(grep source_url $file | sed s/' *source_url: '//)
_blueprint_update_debug "-> source_url: ${blueprint_source_url}"
# check for a value in source_url
if [ "${blueprint_source_url}" == "" ]
then
echo "-! no source_url in file"
echo
continue
fi
# check for custom source_url (the source_url doesn't exist in the source file)
custom_source_url=""
if [ "$(echo "${blueprint_source_url}" | grep '^custom-')" != "" ]
then
_blueprint_update_debug "-! custom source_url set"
custom_source_url="${blueprint_source_url}"
blueprint_source_url="$(echo "${blueprint_source_url}" | sed s/'^custom-'//)"
fi
# fix source if it's regular github
if [ "$(echo "${blueprint_source_url}" | grep 'https://github.com/')" != "" ]
then
_blueprint_update_debug "-! fix github url to raw"
blueprint_source_url=$(echo "${blueprint_source_url}" | sed -e s/'https:\/\/github.com\/'/'https:\/\/raw.githubusercontent.com\/'/ -e s/'\/blob\/'/'\/'/)
_blueprint_update_debug "-> fixed source_url: ${blueprint_source_url}"
fi
# check filename is the same
if [ "$(basename ${file})" != "$(basename ${blueprint_source_url})" ]
then
echo "-! non-matching filename"
_blueprint_update_debug "-! [$(basename ${file})] != [$(basename ${blueprint_source_url})]"
echo
continue
fi
_blueprint_update_debug "-> download blueprint"
wget -q -O /tmp/blueprint_update.yaml "${blueprint_source_url}"
# insert the custom url, if it was in the original, and is not in the newly downloaded file
if [ "${custom_source_url}" != "" ]
then
# check for source_url in the new source file
new_blueprint_source_url=$(grep source_url /tmp/blueprint_update.yaml | sed s/' *source_url: '//)
if [ "${new_blueprint_source_url}" == "" ]
then
_blueprint_update_debug "-! re-insert custom-url"
sed -i "s;blueprint:;blueprint:\n source_url: ${custom_source_url};" /tmp/blueprint_update.yaml
fi
fi
_blueprint_update_debug "-> compare blueprints"
blueprint_diff=$(diff ${file} /tmp/blueprint_update.yaml)
if [ "${blueprint_diff}" == "" ]
then
echo "-> blueprint up-2-date"
else
echo "-! blueprint changed!"
cp /tmp/blueprint_update.yaml ${file}
need_reload="1"
fi
echo
done
if [ "${need_reload}" == "1" ]
then
echo "! reload automations"
hass-cli service call automation.reload
fi
And create an automation to be able to start the update.
you could let it run every couple of hours or days, or only on manual trigger…
it appears to update the Blueprint as well. Feature request be able to run this without updating. I don’t want blueprints updating every few hours by themselves. I want to be there… But I would like to know if the blueprint is out of date checked every couple of hours kind of thing.
It didn’t seem to look in the scripts blueprint folder.
It appears to me that even though they may possibly have a valid filename, blueprints that reside in the blueprints exchange and were imported from there originally do not seem to work. These seem to get -! non-matching filename
My output of the script in a terminal:
/config git:(master) ✗ ./scripts/blueprints_update.sh
> ./automation/aderusha/MQTT_Tasmota_CarPresence.yaml
-! blueprint changed!
> ./automation/hanerd/zwave-js-schlage-lock-user-code-actions.yaml
-! non-matching filename
> ./automation/raz0rf0x/zwave-js-kwikset-910-manual-status-updater.yaml
-! non-matching filename
> ./automation/SirGoodenough/AutoFanControl HA_fan.yaml
grep: ./automation/SirGoodenough/AutoFanControl: No such file or directory
grep: HA_fan.yaml: No such file or directory
-! no source_url in file
> ./automation/SirGoodenough/AutoFanControl_MQTT.yaml
-! blueprint changed!
> ./automation/SirGoodenough/door_open_tts_cloud_say_announcer_nabu_casa_required.yaml
-! blueprint changed!
> ./automation/SirGoodenough/keypad_5_button_cipher_to_turn_on_something.yaml
-! blueprint changed!
> ./automation/SirGoodenough/Zigbee2MQTT - Xiaomi Cube Controller.yaml
grep: ./automation/SirGoodenough/Zigbee2MQTT: No such file or directory
i set this up some time ago, and already thought that the command needed to be made known in the configuration.yaml but didn’t look it up
i’m going on vacation in 2 days, so i don’t have a lot of time to make changes to this script, but maybe i’ll make it into a project after my vacation…
I think what’s going on with the Blueprint Exchange ones is the way the Bluetooth import works. You point it at an exchange post, and it pulls out all the code from it. Therefore there is no actual filename like there is with ones from GitHub. So there’s no actual filename to match. You would have to grab the whole post that it points to and parse that for grave accents or something really messy I think…
in the meantime, i made a couple of updates to my script…
1- fixed, it works with spaces in both filename and source-url
4- i added --debug and --update, by default it doesn’t update anything anymore
5- not sure why that would happen, i changed the file-finding (because of point 1) so maybe it works now
6- if you can reply with a source_url, i can look at it, maybe there is an easy work-around
i also posted the script on gist, so you can get the new version from there…
So the spaces inthe filename was halting the script. I removed 2 blueprints and it got to the end and did the scripts.
Every blueprint exchange based blue print doesn’t find the file, like these:
Also it got to the end of the script now, the automation reload command isn’t functioning.
! reload automations
./scriptshell/blueprints_update.sh: line 88: hass-cli: command not found
Process Completed
(I added a Process Completed echo so that I know when the script was done running manually…)
Also I’m thinking the automation reload command has changed from when you wrote this.
i remembered that i have internet, and can look at the blueprints exchange myself
the few i looked at use a github gist url, so i added a feature to recognise those and make a correct raw url out of them… since by default they don’t include the filename, or the raw url.
the 2 you link to indeed only have the code in the forum-post, i’ll look into that after my vacation, there is probably a way to get just the first post, which makes filtering easier, since home assistant itself also gets the content from the post, so if i search the home assistant code i should be able to find how they do that…
i removed the hass-cli call at the end of the script, since it wasn’t working anymore, and i couldn’t get it to install easily on my home-assistant server… so now it’s just a message that you should reload.
you also needed to make a long-lived access key to be able to do that, so that would only make it a lot more complex for now
next step is probably figuring out how to create some sort of notification that there are scripts that need tot be updated.
maybe that should be handled in the automation, based on a result from the script, i don’t know yet.
that’s why i just never use spaces in my filenames, it’s always such a hassle when scripting… and a underscore looks just as good for readability.
problem kinda is that i find it fun to work on this sort of stuff, but it can eat a lot of time…
i searched how HA does the import, and apparently you can get a forum-post in json if you add .json to the end, and then they just extract the code-block from the first post.
so i implemented the same, just with only jq, grep and sed commands, instead of a html parser…
i tested it with the kwikset blueprint, and that seems to work, i left a debug echo in there which just prints the downloaded/parsed blueprint on the command line, so if you run the script you should see at least 1 correct blueprint
I can just change the names on the ones I own, but I’m not everybody…
The only one I have now that doesn’t work is the one that also doesn’t import from the exchange because the author has it messed up. More than one code block in the top message.
seems that adding custom- in front of the url gives an error, but i already figured i can fix that by just checking if there is a source_url in the downloaded file, and if there isn’t insert the one from the old file…
then there should be no more need for the custom- part
i just made a couple commits, removing the custom- part if it is in a file, and re-inserting it if it isn’t there in the downloaded file.
also made the difference in filename just a notification, and keep on going with the update-check instead of going to the next file
The custom- part was working for me, as Luma (CarPresence, HASPone) does not believe in adding the source url in the blueprint because he is upset that HA overwrites what he puts in there.
I’ll give the new version a whirl shortly.
We’ll see how the new version works.
Everything seems to work perfect.
I would suggest that since many people, like I do, have the folder scripts holding split configurations of HA scripts, another name be used for the folder as your suggestion. I have changed my folder name to scriptshell. This is only a suggestion and you can take it or leave it.
This is awesome in another way. I provide my blueprints formatted so that they are easier to read with the human eye. HA parses and checks it and pastes it like the machine likes to see it, so the templates and stuff gets mashed and hard to read by looking at it.
This actually fixes that…
The readme file is VERY nice as well…
I have a friend that has put together a few HA addons, perhaps this would be a bit smoother functioning and provide better HA interface in an addon form. Or perhaps find a way to meld this into HA core and become part of the core functionality. As you mentioned getting output, the ability to easily schedule this and notify users that an update is available would be the next steps. It really is a big help just like this, however.
In regards to source_url, it was my understanding that that value was inserted on blueprint import, so I removed it from the HASPone blueprints as it didn’t seem to matter (and wasn’t mentioned in the examples). I have no personal opinions about this one way or the other, so if it makes more sense to put those values back, I can get that done right away.
Are you not seeing that value populated on import?
The carpresence ones didn’t have source_url (originally) in the repo, but I had added it in my personal imported copy, When this script came about, I added custom- (for the early versions) and it was able to check for new revisions.
Originally with this script if it wasn’t in the master, and you did not have custom- added to it, it would import via this script without the source line, but rather an exact copy of the master.
Now I think the script behavior has changed, but I think it’s a good idea for the line to be there as then the exact copy in the master will import into the user’s folder.
I don’t have any HASPone but I assumed they didn’t have that line either.
The /config/scripts/ dir is not hard-coded anywhere, you can put the script anywhere you’d like to put it, also when updating it updates the filename which was called as the command.
I guess i should mention in the readme file that the /config/scripts/ dir is just an example.
I’ll look into giving the script a bit better structure, which should help in being able to update just one blueprint, instead off all-or-nothing. I don’t have any experience with writing addons, but maybe i’ll look into that. Help on that part is always welcome, but i guess that we should start a proper github repo for that, since gists are quite limited in functionality.
