One simple solution would be to allow for overwrite on blueprint import. Today, when a user attempts to import a newer version of an existing blueprint, they see the error “File already exists”. Instead, we could ask if they want to overwrite the existing file which would simplify the workflow above by a couple steps.
Ideal approach - automate blueprint versioning
A harder but “better” solution might be something like the following:
Add a YAML directive to the blueprint schema which declares the version number of the current blueprint.
Add a second YAML directive to the blueprint schema which declares a URL for a JSON file (hosted on GitHub) that would contain information about the currently-available release version of the blueprint
A specification for a version.json which declares the blueprint name and a version string
A service which periodically checks the JSON files declared by imported blueprints, compares available versions against the currently-imported blueprints, and notifies when an update is available
A process by which the user can update existing blueprints when an update is made available
Much of what you are asking for has a related working example to use as a start point. HACS.
I would perhaps try to keep the rev contained in the blueprint and try to not restrict to just GitHub, but it would be a nice to have for sure. I’m sharing 6 blueprints in GISTS, and wouldn’t look forward to having to have 6 more repo’s to hold those, but it could be done. Then there are the ones like CarPresence and others that have multiple blueprints in the same repo.
UPDATE 2022-1-11: I moved all my stuff to a REPO now as it is easier to maintain.
I think the HACS model is the right direction. I don’t think one would need separate repos, simply setup the version.json (or yaml or whatever) such that there are entries for each blueprint in a given repo. For example:
Voting for this myself and just adding a note so I can keep updated.
Recently I started getting script errors in the log file and these were related to blueprints I am using for HA SwitchPlate so obviously I need to update these.
As there was no notification of updated blueprints and no easy way to update them, I didn’t know about any updates until I started getting errors and then manually looked.
+1 definately a must have! Actually, when blueprints were introduced they were advertised as the first step with improvements like updating coming later. But they never actually came.
I made a shell script to update blueprints…
I don’t use a lot of blueprints, but figured this would be easier then checking them manually from time to time.
It should probably send a notification or something when a blueprint is updated, but didn’t bother to add that yet.
if you have a script which doesn’t have a source_url in the source file, add it to your local copy with “custom-” in front of the url source_url: custom-https://example.com/source.yaml the script will then use that url, and re-insert the source url after downloading. edit: if the newly downloaded file has a source_url, the custom url isn’t inserted
Create the following script in the /config/scripts/ directory
/config/scripts/blueprints_update.sh
#!/bin/bash
DeBuG="false"
function _blueprint_update_debug
{
if [ "${DeBuG}" == "true" ]
then
echo "$@"
fi
}
cd /config/blueprints/
for file in ./*/*/*.yaml
do
echo "> ${file}"
# get source url from file
blueprint_source_url=$(grep source_url $file | sed s/' *source_url: '//)
_blueprint_update_debug "-> source_url: ${blueprint_source_url}"
# check for a value in source_url
if [ "${blueprint_source_url}" == "" ]
then
echo "-! no source_url in file"
echo
continue
fi
# check for custom source_url (the source_url doesn't exist in the source file)
custom_source_url=""
if [ "$(echo "${blueprint_source_url}" | grep '^custom-')" != "" ]
then
_blueprint_update_debug "-! custom source_url set"
custom_source_url="${blueprint_source_url}"
blueprint_source_url="$(echo "${blueprint_source_url}" | sed s/'^custom-'//)"
fi
# fix source if it's regular github
if [ "$(echo "${blueprint_source_url}" | grep 'https://github.com/')" != "" ]
then
_blueprint_update_debug "-! fix github url to raw"
blueprint_source_url=$(echo "${blueprint_source_url}" | sed -e s/'https:\/\/github.com\/'/'https:\/\/raw.githubusercontent.com\/'/ -e s/'\/blob\/'/'\/'/)
_blueprint_update_debug "-> fixed source_url: ${blueprint_source_url}"
fi
# check filename is the same
if [ "$(basename ${file})" != "$(basename ${blueprint_source_url})" ]
then
echo "-! non-matching filename"
_blueprint_update_debug "-! [$(basename ${file})] != [$(basename ${blueprint_source_url})]"
echo
continue
fi
_blueprint_update_debug "-> download blueprint"
wget -q -O /tmp/blueprint_update.yaml "${blueprint_source_url}"
# insert the custom url, if it was in the original, and is not in the newly downloaded file
if [ "${custom_source_url}" != "" ]
then
# check for source_url in the new source file
new_blueprint_source_url=$(grep source_url /tmp/blueprint_update.yaml | sed s/' *source_url: '//)
if [ "${new_blueprint_source_url}" == "" ]
then
_blueprint_update_debug "-! re-insert custom-url"
sed -i "s;blueprint:;blueprint:\n source_url: ${custom_source_url};" /tmp/blueprint_update.yaml
fi
fi
_blueprint_update_debug "-> compare blueprints"
blueprint_diff=$(diff ${file} /tmp/blueprint_update.yaml)
if [ "${blueprint_diff}" == "" ]
then
echo "-> blueprint up-2-date"
else
echo "-! blueprint changed!"
cp /tmp/blueprint_update.yaml ${file}
need_reload="1"
fi
echo
done
if [ "${need_reload}" == "1" ]
then
echo "! reload automations"
hass-cli service call automation.reload
fi
And create an automation to be able to start the update.
you could let it run every couple of hours or days, or only on manual trigger…
it appears to update the Blueprint as well. Feature request be able to run this without updating. I don’t want blueprints updating every few hours by themselves. I want to be there… But I would like to know if the blueprint is out of date checked every couple of hours kind of thing.
It didn’t seem to look in the scripts blueprint folder.
It appears to me that even though they may possibly have a valid filename, blueprints that reside in the blueprints exchange and were imported from there originally do not seem to work. These seem to get -! non-matching filename
My output of the script in a terminal:
/config git:(master) âś— ./scripts/blueprints_update.sh
> ./automation/aderusha/MQTT_Tasmota_CarPresence.yaml
-! blueprint changed!
> ./automation/hanerd/zwave-js-schlage-lock-user-code-actions.yaml
-! non-matching filename
> ./automation/raz0rf0x/zwave-js-kwikset-910-manual-status-updater.yaml
-! non-matching filename
> ./automation/SirGoodenough/AutoFanControl HA_fan.yaml
grep: ./automation/SirGoodenough/AutoFanControl: No such file or directory
grep: HA_fan.yaml: No such file or directory
-! no source_url in file
> ./automation/SirGoodenough/AutoFanControl_MQTT.yaml
-! blueprint changed!
> ./automation/SirGoodenough/door_open_tts_cloud_say_announcer_nabu_casa_required.yaml
-! blueprint changed!
> ./automation/SirGoodenough/keypad_5_button_cipher_to_turn_on_something.yaml
-! blueprint changed!
> ./automation/SirGoodenough/Zigbee2MQTT - Xiaomi Cube Controller.yaml
grep: ./automation/SirGoodenough/Zigbee2MQTT: No such file or directory
i set this up some time ago, and already thought that the command needed to be made known in the configuration.yaml but didn’t look it up
i’m going on vacation in 2 days, so i don’t have a lot of time to make changes to this script, but maybe i’ll make it into a project after my vacation…
I think what’s going on with the Blueprint Exchange ones is the way the Bluetooth import works. You point it at an exchange post, and it pulls out all the code from it. Therefore there is no actual filename like there is with ones from GitHub. So there’s no actual filename to match. You would have to grab the whole post that it points to and parse that for grave accents or something really messy I think…
in the meantime, i made a couple of updates to my script…
1- fixed, it works with spaces in both filename and source-url
4- i added --debug and --update, by default it doesn’t update anything anymore
5- not sure why that would happen, i changed the file-finding (because of point 1) so maybe it works now
6- if you can reply with a source_url, i can look at it, maybe there is an easy work-around
i also posted the script on gist, so you can get the new version from there…
So the spaces inthe filename was halting the script. I removed 2 blueprints and it got to the end and did the scripts.
Every blueprint exchange based blue print doesn’t find the file, like these:
Also it got to the end of the script now, the automation reload command isn’t functioning.
! reload automations
./scriptshell/blueprints_update.sh: line 88: hass-cli: command not found
Process Completed
(I added a Process Completed echo so that I know when the script was done running manually…)
Also I’m thinking the automation reload command has changed from when you wrote this.
I’ll load your latest and give it a try shortly.
i remembered that i have internet, and can look at the blueprints exchange myself
the few i looked at use a github gist url, so i added a feature to recognise those and make a correct raw url out of them… since by default they don’t include the filename, or the raw url.
the 2 you link to indeed only have the code in the forum-post, i’ll look into that after my vacation, there is probably a way to get just the first post, which makes filtering easier, since home assistant itself also gets the content from the post, so if i search the home assistant code i should be able to find how they do that…
i removed the hass-cli call at the end of the script, since it wasn’t working anymore, and i couldn’t get it to install easily on my home-assistant server… so now it’s just a message that you should reload.
you also needed to make a long-lived access key to be able to do that, so that would only make it a lot more complex for now
next step is probably figuring out how to create some sort of notification that there are scripts that need tot be updated.
maybe that should be handled in the automation, based on a result from the script, i don’t know yet.
that’s why i just never use spaces in my filenames, it’s always such a hassle when scripting… and a underscore looks just as good for readability.
problem kinda is that i find it fun to work on this sort of stuff, but it can eat a lot of time…
i searched how HA does the import, and apparently you can get a forum-post in json if you add .json to the end, and then they just extract the code-block from the first post.
so i implemented the same, just with only jq, grep and sed commands, instead of a html parser…
i tested it with the kwikset blueprint, and that seems to work, i left a debug echo in there which just prints the downloaded/parsed blueprint on the command line, so if you run the script you should see at least 1 correct blueprint
I can just change the names on the ones I own, but I’m not everybody…
The only one I have now that doesn’t work is the one that also doesn’t import from the exchange because the author has it messed up. More than one code block in the top message.
seems that adding custom- in front of the url gives an error, but i already figured i can fix that by just checking if there is a source_url in the downloaded file, and if there isn’t insert the one from the old file…
then there should be no more need for the custom- part
i just made a couple commits, removing the custom- part if it is in a file, and re-inserting it if it isn’t there in the downloaded file.
also made the difference in filename just a notification, and keep on going with the update-check instead of going to the next file
The custom- part was working for me, as Luma (CarPresence, HASPone) does not believe in adding the source url in the blueprint because he is upset that HA overwrites what he puts in there.
I’ll give the new version a whirl shortly.
We’ll see how the new version works.
