The future of YAML

OK, I’m going to try this once again.

  • There is value in both config approaches (GUI and textual).
  • The two approaches address different users and use cases (with some overlap).
  • Work arounds, while they certainly do exist, are not the optimal solution nor are they what people are looking for. Users are looking for a practical and well documented solution that works for them.
  • Integration developers shouldn’t have to choose which approach to support, its a false choice.
  • Integrations should declare WHAT config they need, not HOW to obtain it.
  • Please leave open the possibility for future innovation and not unnecessarily constrain the solution space.
3 Likes

me too;-) would warrant a dedicated button in the read section of configuration/YAML configuration reloading
Reload Inputs
Nice! is that one for @Bram_Kragten?

1 Like

what doesnt work for packages? My whole setup is in packages…

I know that…for now at least. But we were also told yaml for other things wasn’t going away either…so there’s that.

But mostly my point was to provide an example (and as @tom_l has kindly pointed out there are many of those examples with no replies) to show why the inexorable trend toward UI editors as championed by many here isn’t all it’s cracked up to be.

So, as to the example I used above for my experience in using the UI to configure HACS that then later ended up needing it to be removed due to a failed front end and therefore no way to remove it properly from the UI, how is that situation “officially” supposed to be handled since we aren’t supposed to edit the storage files?

If it was configured in yaml I could have just easily commented it out of my configuration.yaml in a text editor and restarted.

that is an honest question. If this is the direction we are going then the devs need to provide a way to fix situations like that. And before anyone mentions “safe mode” it didn’t help me out.

1 Like

You took the less important part of my comment and centered your answer around it.

The answer to your question

Is already on the comment you are answering to, so I’ll just quote it here:

That is my actual question.

Let’s call it GIT friendly/humanly readable.

1 Like

@danielo515 Not sure what you are trying to do. But the ADR and blog are clear on the subject I would say.

If that course for integration configuration (only for integrations that communicate with devices or service), collides with your use case, then I’m sorry to hear that.

Sorry, I don’t understand this sentence. Seems like some portion of it is missing. Would you mind to rephrase?

Yeah edited it, my mind was faster than my hand on the keyboard for a moment :sweat:

Looks more like you’re just lying and then backtracking.

Bah, it’s been obvious for a couple of months that HA is a totally dead-end project anyway. I have no idea why anyone not being paid to do so would waste their time on this sinking ship at this point.

It’s breathtaking to see the rampant hypocrisy of claiming a goal of “making things easier for normal end users” while introducing breaking changes every release and major ones every month or two - many of which require a significant investment of time and understanding to fix.

2 Likes

This is your opinion. Others might differ. No one is forcing you to use it or be part of the community.

1 Like

@frenck- I think the part that @danielo515 is referring to with git (or at least I understood it so since I am in the same boat) is that when I modified something in the UI which then updates .storage is not easy to track in git:

  • either I put the .storage in git (I can live with a private git so ok with passwords in .storage) but then I get to much noise in git due to the state info (such as last login) in the .storage/auth
  • I do not track .storage and I loose when I changed something

I am ok with the .storage not being intended to be edits but humans (at least they are debugging…) but my memory on what I change is limited so I need git to help me :wink:

3 Likes

Excuse me?

Exactly! See blog post “Breaking changes” section.

2 Likes

I’m in a similar boat to you, up until now my entire HAC config is in a private Git repo with the .storage folder in my .gitignore file so it doesn’t cause things to get out of sync.

I have now set up my Pi so it can now git push so I am able to back up the files that keep changing in the .storage folder, and am currently using NodeRED in order to do an automated git push every night at 1AM.

While this sort of works, it isn’t ideal, I do still get the file conflicts which have to be manually resolved, so I’m still searching for a better way to do things.

I’m not running the full HA with snapshots feature, but could probably do something similar by Zipping up my config folder, but I then I lose a lot of nice things that git version control has.

1 Like

Please take a breath and read a lot more about what is going on before you make such a statement. You may not like the changes and you are free to make your opinion known but I find the developers honest and trying to make the best system they can will the resources they have. I don’t always agree with the direction and or answer. I state my opinion but do realize that their answer is final. Sometimes a final answer changes due to resource changes. This does not mean anyone is lying. Obviously you have never worked in a dynamicly changing environment. If you did you would understand a truth today may not be the same tommorow!

5 Likes

If you are doing pushes only from the HA, not sure were the conflicts come from.

I had for some time automated git pushes but stopped doing so because they I was not adding comments to the git commit. Now I combine an automated backup from the whole folder (I use rsync) + manual git commits when I manually change anything. It is working fine but as said above I get too much noise in my git commits for state changes.

Yeah that’s the worry. I guess we’ll get out the flaming brands and pitchforks if that ever happens. For now I can live with this change.

2 Likes

That can be very tricky though, if not impossible in some cases. I can’t come up with a good example from the top of my head, but integrations may look like they require similar data, but in some details they actually need some extra “seasoning” to clearly communicate what configuration options are possible. Let’s call it conditional configuration or relational configuration.

An example from a totally different application (Remmina, a tool for remote access using different protocols):

  • For every host you configure a host and a port
  • If it’s RDP you can provide the username, password, domain, bit-depth etc.
  • If it’s SSH you can provide the username, password, but instead of that maybe which key to use, and domain and bit-depth don’t apply altogether
  • Then there’s VNC, NX, whatever, all with different requirements

To make one thing clear: such a configuration can be easily done in yaml, provided the user doesn’t use options for a configuration variant where they don’t apply. But the user could add those options nonetheless, possibly causing the things to not work as they should. If the user reads the documentation though, this won’t be a problem. But if not, it’s a potential source for frustration.

And here the benefit of the UI-driven configuration comes into play. It ideally just allows you to configure parameters that are valid for what you are trying to do. Other parameters that globally are valid, but invalid within the feature scope, would just be hidden.

So now to the point I originally wanted to get to: it’s hard, if not impossible, to create an abstracted representation for the configuration-possibilities (referred to as schema) that can be used to validate input, and at the same time do the reverse operation of generating a UI for it. So the WHAT you from your statement lines up with the schema, the HOW is the logic to generate a UI from that schema. And because the HOW is a problem, it just has to be done by the integration maintainer.

To further visualize the problem imagine a non-existent integration for which you always need an ip, a port for variant a and b, username for b and c, and username + password just for c.

So now let’s pretend we could traverse this tree to generate a UI for it:

# BTW, this is what we have right now
integration:
  ip: value
  port: value
  username: value
  password: value

This wouldn’t work because all parameters would always be visible. Let’s try this:

integration:
  ip: value
  a:
    port: value
  b:
    port: value
    username: value
  c:
    username: value
    password: value

This would work, but port and username are redundant. If there were changes and not every occurrence of port and username where adjusted, this could break the integration. So my final attempt:

integration:
  ip: value
  a:
    port_a: value
  b:
    port_b: value
    username_b: value
  c:
    username_c: value
    password_c: value

Now this is nice without any redundancy. But by doing this we loose the ability to exactly derive the validation-method from the parameter name. So some sort of mapping-mechanism that tell’s us that port_a and port_b both have to be integers would be needed. And if a new variant c get’s introduced it could cause a problem if the validator-mapping for the new parameter isn’t correct.

So to sum this up: attempts to solve such problems always come with drawbacks that have to be worked around in some way. So the most stable solution is the first example, because here it’s hard to break anything, regardless of any change, added variants, whatever. But it also happens to be the solution that’s can’t be visualized automatically at all. Or at least not in a way that disallows the user to enter data that he’s not supposed to supply.

4 Likes

@danielperna84

Thank you for the very thoughtful reply! I appreciate the detail on the difficulties. “Schema” is exactly what I’m thinking of, although I don’t have the nitty gritty details or experience with how schema definitions might apply or what their deficiencies might be. Your insight is helpful.

Considering your example, would you really need to configure more than one of the variants at a time? Even if you did, is it too much to expect that some redundant information may be necessary to achieve the result? If one accept’s some redundancy, then it might reduce to a validation problem for each variant. Consider:

integration:
  variant_a:
    ip: value
    port: value
  variant_b:
    ip: value
    port: value
    username: value
    password: value
  variant_b_invalid:
    ip: value
    port: value

I can also see that additional markup might need to be introduced to facilitate things. I can also see that an eventual solution to the textual configuration problem would not be YAML. I know some other technologies that might apply exist, but have no idea whether they would address these issues (e.g. Dhall, or of course Lisp (please no flames)).

So there is:

  • A representative set of scenarios that can be used to validate an approach.
  • How to define the “schema” (technologies that can be used).
  • How to validate user input (seems like this applies regardless of HOW the data is acquired).
  • Representative examples in different textual formats (YAML, etc.).
  • How to generate the UI (e.g. something like what jsonforms does)

I suppose this is not the place to be discussing the details of how this would work. But identifying the issues is a great start. Thanks again for the thoughtful reply.

6 Likes

I’m doing automated pushes from my HA machine to keep track of changes in my .storage holder, but I also do sometimes do a git pull when needed:

When I update the config files, usually tweaks to YAML files, or adding custom components, I use a different machine running Sourcetree to do a git push, then shortly after a git pull and a HAC restart on the Pi running HAC to test my changes.

With some more tinkering of my .gitignore and NodeRED flows I may be able to workaround some of the version conflicts that appear when updating my config folder, as I rarely need to make changes to the .storage folder from the machine running Sourcetree.

Jep - above I was proposing to split the files in .storage between config and state so it would not require tinkering with the .gitignore, ideally subfolders but did not get any feedback. I guess I would need to ask in the architecture repo - so far I have only done a PR to core, not triggered an architecture discussion.

4 Likes