The future of YAML

Well, after doing a pip install pyyaml you at least can convert it very easily:

import yaml
with open("output.yaml", "w") as fo:
    with open("input.json") as fi:
        yaml.dump(json.load(fo))

The other direction is almost the same. That way you can use your favorite cli-text-editor to modify the file in yaml. :+1:

Edit:
If you like vim, have a look at this: https://github.com/elzr/vim-json/issues/62

5 Likes

I did read the whole blog post. If I say I had dinner, it doesn’t mean that I skipped breakfast :slight_smile:

No, my ‘exact question’ was about migration. If I make a backup, I don’t necessarily have a(n easy) way to restore.

Here’s my request again reworded to try and capture it in a different way that might be clearer.

I want to see a how-to that walks me through how to take a working HA setup from one device to a new one.
I have a Raspberry Pi 4 in the mail and when it gets here, I want to move my config from my Pi 3 to this new system.
Currently, I can git clone my config repo and then copy over the secrets.yaml to get nearly all the way there.
I honestly don’t know how to restore from one system that uses UI-setup config & .storage to a new one. I would like to see the docs on that.

By the way, what I do not want is a comment like “well, it depends what the user wants as a backup system”. If the intention is to make everything more user-friendly, then a user-friendly way to migrate to a new system should be considered as part of how this new way of doing things works.

Is that clearer, @cgtobi?
I’m not complaining or avoiding reading, I’m asking for the kind of help that potentially thousands of others are also interested in. Thanks!

You do a complete backup of the $CONFIG directory on the pi3 and restore it on the pi4.

rsync -avPp $CONFIG/ pi4:/$CONFIG
4 Likes

I think this gets at the heart of it though. Should adding integrations require directions? How many devices/services/software that you buy today require you to read an instruction manual to use or operate?

I just set up Grafana in docker without looking up any directions, and it works great, and the UI allows me to do whatever I want. Integration flows allow users to be guided through, without the need to lookup YAML directions (which they should be able to follow, but still). I think the points about separating config entries (in .storage) and the data/states is a good one though, that would make it safer for people backing up those files and keep “live” data away from what should be relatively static data.

In the end, I think this is a good compromise between flexibility and usability, but that feature parity should be kept (options for integrations available in YAML should be kept in the flows).

1 Like

Why is it needed? Probably the same reason that you choose to help others learn HA - because you want more people to be able to enjoy the thing you enjoy.

But if Home Assistant was easy enough in the first place, you wouldn’t have needed someone to help guide you, and you wouldn’t have needed to teach someone else. Why put the burden on the community to support each other when it can just be made simpler in core? That is the entire point. The problem can be fixed at the source.

When your initial experience with something is super frustrating, trial and error, making mistakes, most people just give up completely and decide it’s not for them and not worth their time. First impressions are extremely important. It’s not a good idea to overwhelm someone right off the bat.

And the fact is, you can’t even really create any meaningful automations until you first have integrations in your setup. You cannot automate your lights if you’ve never completed step 1 of integrating your lights. That’s why I consider integrations one of the most crucial aspects to really provide a good experience on, because so much of the system depends on them.

If we can get these new users set up with all their devices and services quickly, they are much more likely to ride out that initial frustration / learning curve, because at least they feel they’ve already made some progress. And if they have some basic automations done in the GUI / Automation Editor, there is less pressure to feel they have to learn everything YAML on day one, they can be eased into it over time.

2 Likes

Jesus I have had secretaries who cannot use word properly. “What is a style?” “I can’t get this numbering right” “What do you mean ‘did you save it as a document or a template’?”. These are meant to be professional document creators and they use word like a glorified typewriter. Don’t expect anyone to read instructions. Don’t expect people to do any more than it takes to get through in the easiest way possible. Don’t even be surprised if they use twink on the bloody screen!

I don’t necessarily have a problem with that. Perhaps someone who isn’t willing to do the reading and the work should go with something like SmartThings. My initial experience, along with thousands of others (I’m sure) was super frustrating, but you learn and you get help, and from the help and learning you get better and can achieve more by yourself and you build relationships with others in the community. That’s a good thing.

HA doesn’t have to cater to everyone, and shouldn’t. That was never the goal of this project. The tag line on the home page states…

"Open source home automation that puts local control and privacy first. Powered by a worldwide community of tinkerers and DIY enthusiasts". That last part is key - this project has grown and become what it is today because of tinkerers and enthusiasts, not from people going click…click…done.

I don’t believe that any features should be removed because someone can’t follow documentation telling them how to integrate their Yeelight, Wemo switch or Hue Hub.

Read. Learn. Make mistakes. Ask questions. You will be better for it.

8 Likes

It’s not just about that though, it’s about easing the burden on future development, and avoiding breaking changes in the future (which cause huge problems for the community, power users and newbies alike) at the expense of people who like to backup with git instead of snapshots.

One thing I’ve noticed in this thread a few times is that people are against the UI because changing a setting for an existing integration such as an IP address involves going into .storage to fix it (or removing the integration and adding it again). I’d imagine it would be possible to add the ability to change settings in the UI for an integration that’s already set up so that going into .storage wouldn’t need to happen.

You already can with many of them by clicking the icon in the upper-right, it’s up to the integration author to enable it though.

That’s not what this is about though, it’s about being told YAML will never be removed, and then it gets partially removed. One starts to wonder what will be the next thing that is removed under the guise of “easing the burden on future development” as you have stated, or similar reason.

I keep going back to it, but the only motivation I can see for placing things in hidden folders that aren’t supposed to read by humans, removing long used features and making things so simple that my Dog could set up HA, is money. I’m yet to be told that is not a motivating factor.

2 Likes

I think it’s more about managing the future and keeping things sane rather than money, I’ve never gotten the feeling that Paulus or anyone else working for Nabu was in this to get rich…

If they are, they’re doing it wrong.

2 Likes

LOL :slight_smile:

I didn’t know what that was so I googled it on my work pc :man_facepalming:

At first thanks for original post. Well written, easy to read and understand, well organized covering all needed aspects.

I understand arguments of both sides. Especially for pro users, GUI can be slowing down factor. I guess there should be tools allowing non standard tasks like bulk changes (ie removing 200 devices at once etc) or other other advanced operations over the storage.

GUI is good thing to have but must be reliable covering all needed/basic features. I have in my mind how I struggled with removing entity from the system or renaming it (I mean renaming entity, not it’t friendly name). I know it is not alway up to visualization layer itself, but in a component responsibility, but the difference is something a user doesn’t care if a feature doesn’t work.

Another thing is GUI organization. Currently, there are parts which are simply counterintuitive. like entities customization. Who the hell wants to jump between entity popup and customization page just to set icon? I’m assuming it was easier to reflect data layer directly, without any further transformation to visualization one. But this is obvious mistake often done by programmers: gui is to provide consistent and intuitive experience to a user. Not to visualize data.

Ahh and please revise GUI design / implementation. I don’t know what monitors you are using, but the size of gui elements is too big. I really don’t need to have 3cm big letters. Simply reconsider css styling to make it more convenient of various display sizes.

As a recapitalization: I’m for announced change but would expect improvements in GUI and extra tools for advanced users.

with regards

Fair enough. Point taken.

Restoring from backup works just the same way, as @nickrout explained one possible way, you copy you .homeassistant folder. That should be it.

Again, there is a difference between having a full backup and just some share files on GitHub.

I backup to my private git repo and that’s about it. Of course I also have a backup of that self hosted git server etc.

This thread is unfortunately not the ideal place to discuss backup strategies as it easily gets lost in the noise. So feel free to open a new thread to share ideas.

1 Like

I have to say that I find myself agreeing with an awful lot of what @kanga_who says in this thread.

This project (listening to the story Paulus recounts - in ‘State Of The Union’ I believe) was created with an almost absent minded desire to play around with automating a light - I’m sure that idea sounds familiar to you all :slight_smile: .

So all this talk of expanding the audience and making it easier… Why do ‘we’ even care about that? (I ask this as a philosophical and very high level question.)

I don’t suppose ‘beginners’ are on the whole likely to ever contribute to ‘The Project’ and that is of course more than OK, but where is the benefit to ‘The Project’ in having simply a sheer weight of numbers of ‘passive’ users if not to contribute to the income stream?

And again I say, how is basing the income stream on Amazon and Google, putting “local control and privacy first”?

I am afraid it is now not a rhetorical question. I’d like an answer. How do you reconcile those two things?

(For the record, I don’t actually care one bit about it because of course I/we all have the option of not using Amazon or Google and of not subscribing to NabuCasa. All the brilliance of HA will still be available to me - yes, for free.)

I am not making any judgements on the principle of making money out of HA I just think there needs to be a bit more openness and recognition that this is now the case.

And finally,

Nor have I and I trust that that is true, I actually don’t question their motives at all, but let’s just acknowledge that with money, everything changes.

And getting ‘rich’ is a subjective thing. Supporting you and your family doing something you absolutely love makes you richer than the vast majority of the worlds population :wink:

7 Likes

Very true!

Why we care about making it easier is not only to expand the audience but also if not foremost to improve the usability for everyone.

And contribution to the project as a whole does not have to be the form of code, documentation or support of other users like so many in this community do, but also by asking question, raising issues and showing what HA can be used for. The beauty of Open Source is that everyone can use it, can contribute back, but doesn’t have to.

That quote that @kanga_who posted sums that up perfectly in my opinion. The project is driven by geeks like us but it doesn’t limit its use to only this particular group of users.

Great summary!

2 Likes

The one thing that has perplexed me in this thread is what also bothers me. Why are things moving to JSON? YAML is meant to be a configuration markdown where as JSON is more for data transmission. Why are we going against that? Also because of that I do like YAML for us use with configuration files.

2 Likes