After update to 16.0.0 all flows and notes are gone

I actually do use an SSD. This may be a good idea. I also offload my backups using that Google Drive Backup. But am fully reliant on HA’s backups. What scared me is that when I restored a full backup, it wasn’t actually restored to the old situation (i.e. pre 16.x), when downloading them from either Google or HA directly I could not open all files. And iceing on the cake was a restore that just caused HA to never come back up… Eventually found an older backup of 7 days that at least had the flows and restored those.

Edit:
This has me baffled… why I can not open these files. Also still thinking something wrong with my laptop or setup?
Screenshot 2023-11-15 at 21.21.49

1 Like

Not impressed at all, to say the least.

1 Like

Thanks @JBfr! This is as simple as it will get - and it works.
Just be aware that you will revert to the state just before your very last saved edit. So your latest edit has to be re-done.

2 Likes

This is not the lesson.

First understand Why the change.

At the end of the day this change means your flows and such WILL NOW RESIDE IN THE BACKUPS OF NODE RED post 16.0.2 not HA core. So had the bug not happened you’d all be extatic now because ultimately this will eliminate most of the confusion we’re dealing with. It just unfortunately happens at. The worst possible point.

Second.

In a real IT shop if you go to restore and it won’t because the backup is corrupt. Its YOUR FAULT (i have literally fired someone for this VERY thing. Didn’t test the restore process and validate the BACKUP leading to data loss.). In a shop we don’t trust ANYTHING. Not people, not software, not storage. (Hell we call current security methods ‘Zero Trust’) Fortunately there were multiple ways to get to the data. I test my backup about every 4th one (about a month) just by cracking open the tar to verify it extracts. The way HA is built if you have the files you’re more than halfway home. And I never delete the one I tested until another I’ve tested is there to replace it.

So. Full 1 (tested) , Full 2, full 3, full 4, full 5 (test), delete Full 1 (unless it’s my semi-annual or year end.

Technology is never a panacea and sometimes (me included I was WAY off guard for this one this morning) we get bit.

Your lesson is don’t trust TECHNOLOGY without validation.

5 Likes

I strongly suspect the files are corrupt unfortunately. What other backups or things are available?

Home assistant backup files are encrypted in a way that prevents them from being opened by the regular tar executable.

Check out: decrypt-ha-backup · PyPI
A related thread here: Unpack encrypted snapshot files

Woooow what a day ! Server is up again with release 16.0.2 but can’t find my nodes. Do I have to open a backup file, if so how do I do that ?

Win11 opens them natively

1 Like

I’ll give a try, thanks. But looks like my NodeRed backups are wrong, they only do 12kb lol

Just a dummy question…

if I do a full restore with my backup 24h old, it will work right? I will get all my flow back?

cheers

1 Like

If you do a FULL restore that includes your NR config and restart after yes.

Your NR config pre 16.x is currently in your HA core backup and your node red addon is in the NR Addon Backup. So you’d need to make sure BOTH restore. I’d you miss one or the other then you can get into the ‘wheres my stuff’ again loop.

Post 16.0.2 everything you need will be inside the NR addon backup.

1 Like

Nearly there, got my nodes back but looks like my Mosquitto server doesn’t communicate with my NodeRed anymore, damn

1 Like

I got lucky in that the upgrade did not overwrite the .flows.json files or I would have really screwed. The old files were renamed to flows.json.backup or something like that so I was able to recover by importing them back into a new instance of nodered.

My backups didn’t work either. I set my backups to take one full and three incrementals weekly, and overwrite the oldest after the 4th backup, because space is tight. But when I upgraded to 16.0 and it failed, I tried to upgrade to 16.01, then.02 and so on, which backed up the older instance each time. This eventually overwrote the oldest 15.11 backup so I lost it.

@NathanCu being a fellow IT guy myself, I completely agree with you when you say backing up, and testing the backups regularly is important, because we know the consequences of what a failed backup could mean in a professionally managed environment. However, most of the these guys aren’t IT people, so they (including myself) have a certain degree a faith that if something goes wrong, we can simply restore the last known good configuration with a click of a button. As a somewhat poor analogy, when we drive our cars, we don’t do a complete checkup on the engine every time we go for a drive. We have to have some faith that it will get us to where we need to go. And in such a non-professional non-production “hobbyist” environment, that is the way it should be. Things should just work or at least have a way to recover easily if it doesn’t.

HOWEVER, and not to go too hard on Frenck (or whomever created the Nodered addon),publishing a flawed release of a software with such a disastrous consequence as causing a virtually bricked automation system that so many people have come to rely on is absolutely “unprofessional” (for the lack of lighter term).

Luckily there is a fix for this that will probably work for many people, but it could have much…MUCH worse and rendered MANY people without working automation, which many (including myself) have grown to depend on.

I may speak for myself when saying this, but please PLEASE double,triple…quadruple test your applications before releasing to the public. Many are not IT savvy nor do they know enough about JSON files or accessing the addon share to make changes, nor should they. I’d like to think that HA has evolved to the point where its ready for anyone who is interested in this stuff to adopt it, but after today, I am beginning to have my doubts, and all releases should come with a forewarning of potential damage.

3 Likes

+1 @Mirai_Hayashi

I’m not an IT guy and HA and the professional look that it wasn’t made just for IT savvy people was the reason I decided to use it

@NathanCu If you fire people for that little thing, the person who released the mistake would undoubtedly be fired - although I do believe people learn from mistakes, so I wouldn’t fire any of them.

2 Likes

There were extenuating circumstances (im not That mean) he’d been warned.

2 Likes

Thanks for your time trying to help people here. My circumstance is that the whole folder /config/node-red is not included in my backup files in the last three months. Is there any way to find it in any other places?

1 Like

My HA instance does local and Google drive full backups daily.

I selected the Full Backup of HA on whatever day it was that NodeRed was pre 16, and did a full restore to that (it was only a couple days ago).

My HA instance came back online while out doing errands.

I returned several hours later to re-read this thread to proceed with an attempt to update NR to 16.02

NR was running and I exported all flows to json.

I then went to HA Add-Ons to initiate an update, only to find that NR was already at 16.02

It had come back online with the 15.x version and proceeded to auto-update to 16.0.2 (apparently skipping past 16.0.0 and 16.0.1)

I didn’t need to touch the addon_configs directory.

1 Like

If you have a node red folder in your addon config folder (sorry I can’t remember the right name right now) the rename on-don’t delete it it probably has your flows. There’s a way to import it apparently (someone mentioned it in one of the responses this morning)

Maybe?

somehow it is included in my node red 16.0.1 backup file, which I tried to upgrade to 16.0.2 and never thought of opening it to look for this file.
It’s renamed .flows.json.backup, I hope this information can help others who couldn’t find his ‘flow.json’ in previous full backup files.

1 Like

Same here!? AAARGH! All gone! What!?