The future of YAML

insert gif of stickman with ‘the point’ flying over his head here

Looking at absolute numbers alone is inadequate, since it wasn’t always this way. To get a better view, one would have to e.g. plot new contributers over time and then add markers which indicate philosophy changes. This of course would need to take a deeper look at the nature of the contributions. Changing documentation and/or three lines of code isn’t whats relevant here.

1 Like

Just decide if you want to stay with the HA if its not going in the direction you want it to and accept it or leave. Its your choice

As HA is still not at V1 then I’d consider it beta, and changes of direction are pretty much ‘part of the game’.

I’d like to think there are more options than accept or leave. As this is open source, I’d like to think there should be a ‘fix’ option too. :slight_smile:

2 Likes

I strongly do not agree with that. I didn’t know anybody, heck, I was bored to death when I found out about Home Assistant back in 2017. My newborn and wife sleeping; Me sitting there, trying not to make any noise, browsing the net.

I’m not worried about the culture of any of the HA team and not worried about the general culture with the contributors.

I wonder where you get that feeling from? What have you tried to add to our codebase to come to that conclusion? Did a PR go south?

I know one thing for sure, I did not go into these forums complaining about theories on an inner-circle or a culture. Nor did I address developers personally as you did with me just now…

I started coding, writing and helping out. I’ve spent around 30 hours a week, besides my full-time job and family with young kids. I still make 60+ hours a week, I’m not sad about it; I simply love doing it.

As that is part of my motto I stick to with every day; “How am I going to make a difference today?”

But it is things like that response, you wrote there, directed at me personally that doesn’t make me the slightly assholic right now.

But that’s just my 2ct. Since, while I work for Nabu Casa, I’m still a paying customer myself. Just because I feel like I want to support the project. Feel free to completely ignore me.

…/Frenck

10 Likes

@frenck It’s a good response, honestly :+1:
I like that you replied calmly and in many ways I agree with your points and share your feelings.
Imho it was not appropriate to address you personally and publicly in such a way, it’s far beyond but that’s their choice I’m afraid.

Just wanted to add my 2ct here as I can’t say I totally don’t understand that person’s feelings.
So I want to explain a bit.

We’re all different and at different point in our life we evaluate things through a prism of our past experience, feelings and a lot of other things. So even one person might react to the same situation differently today than they did a year ago. And a year later it might be not the same as today. Completely normal as we’re not robots.

That’s why we often fail to understand other’s motives correctly - it’s just impossible to get into their shoes, especially if the only thing you know is some words that person wrote as it’s just words, they don’t always directly reflect or accurately convey person’s feelings (as not all of us are great writers or psychoanalysts) and hey, you need to take into account that for many of us English is not their native language and it makes things even more complicated (don’t forget about culture differences, too). So it’s often “lost in translation” and it happens to us all every day in life but here it is more prominent because of the nature of the interaction - in 99% of cases you don’t know a person you interact with and the only thing you know is the words on your screen (and maybe your past interaction with that person if your memory isn’t full of other stuff already).

Because of that, your feelings about culture in HA and the general culture with the contributors or whatever else are you personal an unique traits and cannot be compared to anyone else’s, it’s just pointless because of the above and at least one more important thing: you have a special role now.

Imagine you’re the secretary of state/president of your country. You usually act not as an ordinary citizen but have to consider the country’s interest and in many cases that means your actions impact life of many of your citizens in a hope that the country as a whole will benefit from it (maybe in a long term and after a period of hardship). There is no way to please every single person. And, very importantly, you’ll never fully understand how your citizen feel about your actions as a) they don’t know what you know about the situation in the country and b) you don’t know how it is to be an ordinary citizen (especially after some time in the role, you’re more or less living in a “bubble” and don’t really know and understand what’s going on on the “ground”).
A good example is the current coronavirus situation. People of many countries are forced to stay at home as the government wants to stop spreading of the decease and prevent their health care system from overload. To make sure it works some strict and sometimes stupid-looking things are enforced. For example, one can do their physical exercise outdoors with a person they live with but it’s not allowed to play tennis with the very same person. From your point of view it makes perfect sense as if you start making exclusions, it won’t work. On the other hand, a lot of people will feel really annoyed/angry because for them it’s just stupid - they do everything together and it won’t do any harm if instead of a walk hand-in-hand they play a game. One situation, one goal but very different feelings.

Now back to HA. Currently you are not an ordinary HA user. In many ways you’re different from all of them.You can directly influence the way it goes as you can write code, make your point during internal discussions and stuff. You get much better understanding of the current situation and many big and small details that are all part of the complex puzzle called HA. And your “How am I going to make a difference today?” has a special meaning because you really can make changes.

But the picture is different if you try putting yourself in the shoes of a normal HA user (preferably not Frenk v.2017 as you’ve become a member of the HA team because (I presume) you suited each other well and the situation was right for you personally).
A direct analogy with coronavirus situation described above. You’ve been told to do this or that, sometimes with a stupidly sounded explanation or even without. And you actually have no choice as there is law, police etc. You can obey, start drinking, go to jail, commit suicide… that’s about it.
Fortunately, it’s not as bad with HA as one can leave the country but it’s hard after years in it and thousands lines of configs so I really feel for those who have to do that, it’s tough considering they loved HA when they joined, and they spent and possibly contributed their time and knowledge but now are forced to make a move.

You ask " where you get that feeling from". I presume it’s not only about PRs going south (it happens), it’s more about the way of interaction between contributors and decision makers. And for that reason it might be difficult for you to understand it fully as you’re in a different camp/play a different role. It also might explain why you are not worried about the general culture with the contributors - you are no longer one of them (as I explained above about your role) and things have changed dramatically since you were.

My personal (a very modest comparing to the others) experience was similar. I saw situation when myself or someone else from the community made a PR or opened an issue that affected many of us (like issues with away_mode in generic_thermostat - the issue is still there after more than a year) and yet there was no response/actions/review or the response was shocking an inadequate (the most notable - from awarecan, something like “I won’t spend my time dealing with a component I’m not interested in” in the middle of discussion).
With time and some successful/not so successful contributions I kind of developed a view that allowed me to carry on contribution even if my contributions/issues were refused for some weird reasons or completely ignored/misinterpreted (lack of the core devs’ time or a more deep weakness?)
I made some contributions and I helped people on the forums by sharing my experience/code and I’m glad that some of them were able to benefit from it. I know there are people who see my participation in discussions wrong for various reasons but that’s another story.

This situation made me look at HA from a different point of view and ask myself a slightly different question.
I’m an ordinary user here.
Do I really want to spend my family/sleep time writing code/making a contribution that could be rejected/put on hold/not looked at as there is no clear and predictable process?
Do I really believe I personally can make a difference by contributing?
Do I think it is sensible to build my smart home solution around the product which one day might change its nature not because of its users’ will but for some obscure reasons I’m not aware of and that would force me to decide whether to stay or to go?

Unfortunately, I have no positive answers here. Mainly because I’m not convinced that HA is community-driven project as its community has no say, it isn’t involved in the decision making process as we can see.
Don’t know if you personally understand that and don’t think it will impact anything in HA world as I’m just one in an ocean of users and it won’t matter whether I stay or go or whether I continue contributing on not.
On the other hand, if each of us at some point will ask themselves similar questions, it might have quite substantial impact as this project relies on community. From the history I know what happens when people in power get too detached/(self)isolated from who they rule.

12 Likes

Lower motives should must be the last possible Explanation evaluated. Not the first.
That’s what I’ve noticed in the PR you referring to as well, but that’s certainly not the only occurrence nor the reason why I think about the situation this way.

I do understand why people who work with end-users a lot may fall into this behaviour, because indeed these people tend to be like that.
DEVs however usually aren’t. And that’s where things start to break.

I know it’s hard to handle these different user bases, since you cannot for sure know, whether or not the person you’re interacting with may be problematic. I’m just worried about the defaults.

This shouldn’t mustn’t be the benchmark for newcomers.
It’s nice that you do that and I too have spent countless hours for FOSS, but to require this kind of investment into this one specific project as the criterion which defines whether or not opinions are valid, isn’t exactly an open project.

Edit:
Also, 100% what @akasma74 said right above this comment.

3 Likes

I still disagree since just by looking at the past 20 releases there were on average more than 80 devs contributing to HA. If you consider 80+ the inner circle… I don’t think so.

The number of integrations is still growing with every release and this is certainly not driven by what you call the inner circle, let alone the “paid devs”.

Yes, reviews can feel discouraging and harsh at first, if that is what you refer to, but this is common and in hindsight usually improves the code quality.

2 Likes

I think a lot of people commenting on here have been in isolation too long. Look at what is currently going on in the World and put things into perspective. Chill out and keep healthy.

6 Likes

It always pays to remember Occam’s advice.

There are two facts, which, in combination, cause imho catastrophic results:

  1. Currently, the documentation leaves much to be desired.
    That’s of course an issue that I completely understand. It is a lot of work to properly document a project of this scale. Especially since everyone who could do that knows it so well that documentation isn’t necessary for them. Thats why many projects commercial as well as FOSS lack documentation.

  2. Currently, reviews are as you’ve mentioned rather harsh and discouraging and getting answers to questions you might have is a tedious task at best. If you can get an answer at all.
    Of course that’s also something I completely understand. There’s limited amount of time available and it’s more fun to spend that doing something you like, improving the software etc instead of interacting with newbies.

Problem is, that of these two things, only one may exist simultaneously. Otherwise people will fail to meet expectations, since they can neither look up the answer by themselves nor ask someone.

They just don’t have any other choice. It’s impossible for them to prevent unpleasant PR situations.

The only way to avoid this experience is to not contribute at all.

2 Likes

Well, I give now my comment on that, because there is complete wrong attention to it.
I think some did not read the full Blogpost, mostly miss the needed technical background behind that text.

First, what changes to now? Yes, nothing. The state now is, a YAML device config generates a full automated config in the background. What every you want rollback with YAML, did today not work if they use the new device/backend API, which is a lot integration today.

Because people had in on YAML, it implicit that they work as before but it did not. There was no maintainable solution for developers to make the UI user happy or the YAML user. We use the hybrid YAML/Backend (Yes, it’s not a UI config, it’s a backend thing) on many stuff like Automation but it doesn’t work for device configs.

But the problem is the community that makes the need of this change now happens. If a developer spends time to make an integration with UI/backend support, the YAML people complain, otherwise, the UI people complain. That generated all-time fire to the developers because they can’t be doing that right. Also because a device config is mostly once on an install, all the YAML fans did not register that the YAML did not work like on old days now over a year.

What we now do is remove a not working layering and clarify the situation for everybody.

3 Likes

I understand your frustration with this and the backlog of almost 200 open PRs wont improve that situation. I can just encourage you to be patient.

I think this discussion slightly derailed from its original topic which was about the future of YAML.

3 Likes

Personally I didn’t mind the elitist approach of “do it in text files like this even if it does scare off newbies who can’t read simple documentation”. But I have been configuring linux from text files since about RedHat 5.

As someone said earlier in the thread, dumbing the process down (by making it it all UI and clickety click) just brings in more dummies. And then they need supporting, because frankly you can never dumb it down enough, no one reads documentation, and even if they do it is never simple enough. No one reads the introductory Howto for the forum How to help us help you - or How to ask a good question, no one understands how to type in a terminal (WTF, if you can type in a forum, you can type in a terminal), everyone watches out of date youtube stuff, and with rare exceptions the youtube creators never post an update saying “this video is out of date, I’ve done another one, see here)”. Some people can’t even copy and paste text, and insist on posting pics of their text files (good luck to thise searching the forums for a problem if the problem is in a jpeg!).

So in my ideal world, we’d stick to the geeky text (yaml) configurations and deliberately drive away the non-savvy people. No problem to geeky me.

But I am not the only person out there. And my background as a commandline user is not the same as most people. And it ain’t my project, if I have contributed anything to github it is docs only, and if the devs want to go another way, I am gonna stick with them for the foreseeable future, because it is still a great project. And I’ll try and educate the people with skills different to mine, and hopefully we can all make home assistant even more awesome.

I must confess I set up a supervised HA in ubuntu on virtualbox this afternoon. Smooth. Lots of stuff detected on my lan and easily incorporated into the setup. I am impressed. I may even move my own setup to that instead of core on docker. It is a long time since I started from scratch, and it sure will be helping those newbies. All power to the devs.

Please community, let us keep calm and carry on.

16 Likes

I’m not exactly sure what you mean by this to be honest. Care to explain how it’s an “ugly workaround”?

I will give you a specific real-world example using the Apple TV integration. This is the YAML config for that integration:

apple_tv:
  - address: 10.0.0.1
    identifier: <id>
    name: Living Room
    protocol: MRP
    credentials:
      mrp: <creds>
      airplay: <creds>
      dmap: <creds>

Are you going to remember the IP address of that device off the top of your head, or are you going to have to spend time looking it up on your router admin? Would anybody know how to obtain the identifier and credentials? Probably not, so it’s off to Google to find documentation on this. Do you know what generation Apple TV device you have, because depending on that the value for “protocol” will be different. Now that you figured that all out, time to restart. But maybe it’s not working because you had a minor typo in a 32 character ID, so time to restart again. Etc etc. This is too complicated and time-consuming for most people. Not a great user experience.

When you set it up from the UI, it scans your network and automatically detects the device and has the field pre-filled with this info, you simply hit the submit button. And it works immediately with no restart. If you ask me I think the UI way is far more elegant and user-friendly. And thus it would make sense to replace that YAML code block above (9 lines) with this single line when sharing your config with others:

# Set up Apple TV integration in UI from Configuration -> Integrations

The point I was trying to make is that you can still get all the same information across when sharing, it will just have to be done in a slightly different format with a YAML comment(s) going forward. I don’t think this is an “ugly workaround” personally. Almost everybody who shares their config already uses YAML comments (including the poster that I was originally replying to @anon43302295 before he deleted his awesome/well-documented repo last year).

4 Likes

Small disclaimer; this response talks about contributing in ways of code/documentation PRs. Yes, we have more ways to contribute.

I wanna thank both @Hypfer and @AhmadK for their good and clam responses on my personal story. :+1:

Thanks guys!

It is my personal story, but it is not a benchmark. I want to clarify that for sure. It is now about the hours you put it, it is about the commitment one has to the project. That makes someone a core contributor. Consistency is part of that key, not hours or quantity.

We have a lot of contributors that don’t have that amount of time that I have (and I’m crazy enough to sacrifice), which is in no way a requirement.

Reviews can be harsh but are never personal. That is the whole point of PRs: Asking questions, maintaining the quality level (and raise it). Same with documentation, it became a lot stricter. And if you could compare it with 2 years ago, it made major leaps. I merge around 200-300 documentation contributions each release… @AhmadK for one, is contributing regularly small tweaks to the documentation, which I absolutely admire him for. So thank you for that sir! :heart:

IMHO, it always boils down to my motto: "How are you going to make a difference today?"

For me, today, that will not be the forum today, as this topic has been highly demotivating for me as a contributor.

Instead, I’ll be finishing what I started yesterday: Rewriting the Almond add-on. Because it needs an upgrade, I’ve migrated it to the new S6 overlay, and make it a couple of hundred MB smaller in size. Why? Because people use it, and saving disk space on someone’s SD card can make a difference for them. Not me, I have 12Tb in my machine available.

I’ll be opening a PR, and will get the same treatment by (probably) @pvizeli on the PR review. Maybe I’ll learn a thing or two, or he is able to prevent a bug from sneaking in.

That is how I am going to make a difference today.

…/Frenck

7 Likes

@SeanM

Oh I see…and how will that work when it comes the time to setup the Modbus UI Integration? Not forgetting the Modbus Switch, Modbus Binary Sensor, Modbus Sensor and Modbus Climate UI Integrations? Not forgetting that in many cases (for many reasons that require some comment notation) you will also require Template Sensors. Will Template Sensor UI be coming as well?

And the reason why Itead have also begun releasing Sonoff branded Zigbee devices

@frenck any advice on how to handle these 3 files?
I committed all 3 yesterday, and all of them have changes since…

:man_shrugging: I don’t use Modbus integration and have no answers for you on that, you’d have to ask the codeowner of that integration about their future plans.

1 Like

What? Not sure what you are referring to.