After update to 16.0.0 all flows and notes are gone

I did not manage to get NR running after restore, and not after updating from that (no clear error message. I tried reverting to a backup (temporarily copying flows), removing node red, reinstalling node red from scratch, rebooting and then it started working and I was able to import my flows again. Did take me quite some hours of trying all possible scenario’s, the ones mentioned above did not work for me. Restoring a backup (FULL) still resulted in NR not working

Where is now located all config files from nodered? Before on config/node-red i can see the files. Now that folder is empty

THe answer is in this thread starting at response #9

Either restore to a pre-16.x node red add-in back up AND core backup tha contains the /config/node-red folder THEN upgrade back to 16.0.2 OR find a backup that contains the files and import them. Both paths are detailed above.

Sure glad I run HA as a VM that’s backed up daily to external media. I restored to the day before the 16.x updates, then updated directly to 16.0.2 without issue and got all my flows back.

You can install Node Red directly outside of Home Assistant - for me, Node Red is more critical to my smart home than Home Assistant, so I don’t run it as an “add on”. I have automations setup in Node Red with mqtt that don’t even go through Home Assistant. You can run it in docker or via several other ways.

Docker:

Other install options:

https://nodered.org/docs/getting-started/local

Backing up in docker is just making sure the volume is backed up - which is just a file path on my Ubuntu machine. For me that’s backed up to a separate USB hard drive, NAS, and google drive, using Ubuntu’s backup program (which is deja dup). I have never had an issue with an update with a Node Red package directly from Node Red - Many well known companies use and rely on this program far beyond home automation, and the project is highly maintained (it was spun off from IBM). I prefer to install the version from “the horses mouth” so to say. Even still, I never auto update it and do a full backup prior to updating.

I’m sorry to everyone going through this, but I don’t really know what the developer of the addon is doing here by moving the .json and config files around like this. You wouldn’t have this issue if you ran the stand alone version of Node Red (I’m not).

Update:
Ok I understand why the files were moved based on the write up here -

But the issue for node red is it has encrypted credentials and a security feature that makes it so you can’t just take these credentials and move them around to another machine/location without carefully updating and setting a “credential secret” in the settings. The warning is right in the settings file:

By default, credentials are encrypted in storage using a generated key. To
specify your own secret, set the following property.
If you want to disable encryption of credentials, set this property to false.
Note: once you set this property, do not change it - doing so will prevent
node-red from being able to decrypt your existing credentials and they will be
lost.
1 Like

Running HASS with Node Red 15.1.1 and works perfectly. If I update to 16.0.2 I ruin my Alexa-Remote2 integration and it is asking me to go to https:\myhomeassistant:3457 to re-login to Amazon. This failed with “page not found” at Amazon so I’m forced to restore a full backup from the previous day to get it to work again.

Is this because the add-on is old and deprecated? I could always try to upgrade to alexa-remote2-appelstrudel, but I do not want to experiment if the issue still is the same with that fork of Alexa integration. Anyone else got issues with Alexa on 16.0.2 ?

I’ve a newbie question regarding HA Backups,
I don’t understand what is restored when I do a Full Restore Backup and why I should do extra manipulation to get my NodeRed back, I would like to understand.
My question is more general. Where can I find documentation about HA backup/restore mechanism (I tried to google but didn’t find). Any links/hint will be appreciated, TX !

Take a look here

1 Like

I still struggle to get NodeRed back,
I try to recover the files from HA backup file “Full Backup 2023-11-06 07_03_49.tar”
I can untar this file that gives me sub files, see below this 1st step is fine.
Then I want to retrieve the files in (1) “a0d7b954_nodered.tar.gz”. But when I do a (2)

tar -xzf a0d7b954_nodered.tar.gz

I get an error message (3)

tar: invalid magic
tar: short read

Can anyone help me to figure this out ?

Thanks,

I followed this approach and no luck.
My scenario:

  1. Updated to NR 16.0.1 from 15.0. (was already on Home Assistant 2023.11.2)
  2. Once NR update finished, NR did open and got a message that showed multiple nodes not existing message

Recovery step 1:

  1. Downgraded to NR 15.0
  2. Checked contents of addon_config and was empty
  3. Started NR and no luck

Recovery step 2:

  1. Still with NR15 unzipped a .tar backup from two weeks back (at that time I had NR 14.6.3) so that I would have node-red folder from two weeks back
  2. Copied this folder from local disk to Home Assistant, and the copy failed as some files could not be copied

Recovery step 3:

  1. Restored my full backup from two weeks back
  2. NR is not opening, but the log is not showing the previous information that services were being stopped (like when I updated NR to 16.0.1)

Recovery step 4:

  1. Still with previous full backup restore, restarted Home Assistant host
    2, When trying to start NR, I get a 'BackupManager.do_backup_partial' blocked from execution, system is not running - startup

I am getting so frustrated, as I’ve been since the last two days doing the above, as any backup recovery/ copy/ etc takes hours for me

UPDATE: after Recovery step 4, I waited for ~1hour as the system was coming up (I’m running my setup on RPI4, so not the most performant device around) and NR was started, with all my flows in it.

I have since updated to NR 16.0.2 and can start it again :sweat_smile:

I updated the NR add-on yesterday, so I missed all the “fun” of having my configs go missing.

But I’m chiming in anyway because I manage my config directory as a git repository. I consider any config change that I make, including NR flows and yaml to be “code”.

By moving the NR config out of the config directory (which I think got renamed to homeassistant?) any changes I now make are outside of my repository. I can’t track them without making a new repo, or moving my repo up a level (changing all of my git settings).

Just to be clear, I’m not using git as a backup mechanism. I use backups for that. Git provides me with important history over time. I can’t imagine that I’m the only one who uses git to track changes.

Moving and changing server directories has significant consequences. It’s unclear what was driving these recent changes. Hopefully there was some rationale more than just cleaning up the names of things.

Backup and restore

The change moves the backup out of your core backup and into a folder controlled by the addon

Pre 16.x to completely restore NR you had. To restore a NR backup AND a Core backup simultaneously to get all of your NR confi

(even if you didn’t want to restore core potentially restoring ‘too much’ (and you can see from the runaway thread above EXACTLY what confusion that causes… I bet you didn’t know you needed to restore both, it was a misconceptions we were all commonly under…)

So now post 16.x ONE restore of the addon backup ONLY - no need to restore the core backup - will get you back instead of the two it took to fix this issue. And puts behavior in line with common expectations - preventing data loss or ‘restoring too much’

Yes it broke a bunch of stuff alotthe way. (my problem isn’t that it happened. It needed to happen. It’s just they didn’t seem to consider all the other unintended side effects. Such as… Now you have to explicitly call out write able folders in your config (also more secure but very breaking to anyone using NR to handle files under /config)

Incredibly ironically the issue broke it in the very way they were trying to make it better.

I agree with the change of the file location to be in the addon, but to say this update was poorly executed is an understatement, and this should have easily been avoided. This started with the update to change the credentials secret and make it optional. As I pointed out above, there is an explicit warning in the Node Red settings file to NOT mess with the credentials secret, as “doing so will prevent node-red from being able to decrypt your existing credentials and they will be lost”. Second, this should have been caught with even minimal testing.

I have minimal programming experience, and I’m sure the people working on this know much more than I do, but this really shouldn’t have happened. Everyone makes mistakes, but the release notes for this change show a lack of professionalism, and even contempt for the users to a degree, which shows a potential deeper problem for how the addons are maintained. You can read them and judge for yourself here Make credentials secret optional by frenck · Pull Request #1730 · hassio-addons/addon-node-red · GitHub

The core version of Home Assistant is highly polished, with many more developers and testing, so I would not have this criticism of Home Assistant itself, but the way the addons are managed is concerning to me. That’s one of the reasons I don’t use them and prefer to install the source programs in docker and manage them myself.

If anyone else is interested, sure there is a learning curve, but its not as hard/scary as one would think. He has some excellent videos on it:

1 Like

Fair points, and to each their own. However even after this, I totally trust Frenck’s skills at managing this kind of thing far more than I would my own.

This worked for me.

  1. restored a node-red backup to pre 16
  2. copied all from config/node-red that I found in a core backup before upgrading to nr v16
  3. verified that NR pre 16 runs with that restored config.
  4. updated to 16.0.2

the addons-config was already empty in my case.

1 Like

This is not a good change. I was using Node red to grab the flows.json file and emailing a copy to myself as a backup. This no longer works, so I’m left dissapointed :slightly_frowning_face:

This worked perfectly! Thank you!

I appreciate this is a double post of mine but please look at the suggestion in post #45 as it saved me a lot of hassle needing to restore things. May not work for everyone but I had my nodes/flows back in 5 minutes.

Did a restore and then upgraded to 16.0.2 All the flows were back and working. However, I have a Node Red dashboard that has js gauges and this dashboard fails to populate. There is some static text which does appear but the js nodes do not show the gauges. Could this be due to the Node Red data being moved out of HA config?