The future of YAML

With pleasure. Given the fact that I was talking about that migration and you didn’t point out to the fact that we are doing forced migrations without notifications to users (on their system, since not everyone may read the Blog/Community), I thought that it was still worth bringing to everyone’s attention.

They do, though. YAML will create UI entities if the import method is present. Also, apparently, some changes (e.g. host field) may trigger a new UI configuration (I have not verified, but it was stated on the GitHub entry as well).

This is confusing for users for two reasons:

  1. Users may not have received a message about this. I know some components log about the migration/deprecation, but that does not seem the case with Harmony; so probably also not others. And not every user may check the Blog.

  2. Some components still support YAML. As such, a user may change the YAML configuration of one component (e.g Harmony) and it would not work; but changing others may. Without additional context (e.g. checking Integrations, and understanding those are now migrated) or notifications, there’s no way for the user to understand what is happening.

1 Like

I guess this is the part that threw me:

Because the user never made a choice in those circumstances. It was done for them, fairly quietly unless they read the release notes or happen to look in their logs at the right time and see it - if the integration even mentions it in the log in the first place. I don’t know if any or all do.

3 Likes

Looks like we cross posted…good points tho… :wink:

1 Like

@nitobuendia, @finity Honestly, I don’t care. If you have problems with it, make a PR against the docs to make it more clear. I’m not going to debate either of you on this issue.

1 Like

Apparently you did until you got a bit of pushback. Or at least you did care enough to post about it in the first place.

There’s no debate about it at all. It’s simply a clarification and statement of fact.

Sorry to have ruffled your feathers.

3 Likes

Hi @petro

No need to debate if you don’t want. Do note that you entered in the debate yourself, though. I personally do not find it very friendly to enter a debate, then say you don’t care or want to debate :wink: No issues, in any case. Don’t worry.

As for the PR, yes, if you read the GitHub, we are talking about that. Thanks for the suggestion.

1 Like

Not at all, just fed up with the semantics debates. I already made it known that i could have been more clear. Yet here we are, still talking about it.

And I should have been more clear myself that I accepted your mea culpa. So mea culpa. :wink:

I was just explaining why I thought there was a misunderstanding. It’s all good on my end.

2 Likes

I was not entering the debate. I was pointing out what I thought was going wrong. Word of advice: not everyone comments here to debate with you personally. Some people are just trying to help, whether that help is correct or not remains to be seen.

Hi @petro

I am also trying to help those users who may get confused by this forced YAML migration; or with some of the statements that you posted in reply, that at least me (and I think @finity) do not find fully accurate or at least clear.

Some of us do find that debate is a healthy way to contribute and clarify, as opposed to sparing smart statements and leave.

In any case, you’re not interested in debating; you made that clear. We can park it here. I can continue debating and discussing with whomever finds this conversation relevant.

Thank you!

4 Likes

As was I.

And that’s what I thought I was doing…trying to help clarify a point of confusion. Maybe not on your part but possibly on someone else’s part that might come by later and misread your comments.

2 Likes

Wow! we really have to stop cross posting like this. It’s getting really creepy.

1 Like

Can someone expand on this statement? Or at least provide a little more clarity. is this referring to the configuration.yaml file itself, or configuration in general?

Any integration polling data from devices like a hub (Hue) or a cloud based service (SmartThings) will only be allowed to be configured via the GUI.

1 Like

This is a solution (as owners said, do it yourself) for those of you who, like me, stopped updating Home Assistant due to YAML Components. I wanted to share a way to downgrade your components to the versions where YAML was being used, so you can upgrade the rest of the platform.


Introduction

Git (GitHub) keeps version control of all changes to components and you can revert to any version. On this process, we will simply download a copy of the home-assistant/core component at a certain point of history, where YAML was still supported.

As such, the key point is that YAML has to have been supported at some point for this to work.


Process

Before you start, it is advisable that you create a backup (a.k.a snapshot) and download it in case something goes wrong and you need to revert to this point in history.

Finding the right version

The first task is to find the most recent version where YAML was still supported. For this, we are going to go to home-assistant/core > homeassistant > components. Then select the platform that you want to downgrade. For this demo, we will use roomba.

If UI is the main configuration, there will be a file called “config_flow.py”, click it and navigate to History on top right.

Scroll all the way down and check the date of the first commit. If there are many changes, you many need to click “Older” a few times. In this case, it’s April 11, 2020 and the commit is Add config_flow for Roomba (#33302). This is the commit where config_flow.py (UI) was created. We probably want to revert to the version exactly before this one.

Within that view, click now on the name of the platform folder, to go to the History page for the whole component. Your objective now is to find the commit before where config_flow.py was added. Again, you may need to click “Older” a few times. Memorizing the date before will help you in doing this faster.

In this case, we want to revert the component to “Add prettier (in pre-commit and CI) (#33693)”. The last version before UI was added.

Downloading the component

Via Command Line

If you are familiar with git (GitHub), you can clone the repository at that version (example):

git clone https://github.com/home-assistant/core.git
cd core
git checkout 39336d3ea367e5c50e88fa09165257290a07b150

Where 39336d3ea367e5c50e88fa09165257290a07b150 is the commit id. You can get it by clicking on the commit.

Once done, copy the platform folder (e…g homeassistant/components/roomba). See next section to know what to do with it. Once you have copied the folder we need, you can delete the overall folder containing the other files of the home-assistant/core repository.

Manually

If you are not familiar with git or do not want to clone the repository, there’s also a manual way:

Once you have identified the version you want, click the button [<>] to revert the view to that commit/point in history.

Once you click it, you will be back to the main home-assistant/core folder, but with a key difference. Now, you will be in the view of the repository in the point of history where you selected.

Navigate again to your component folder: home-assistant/core > homeassistant > components > (e.g. roomba)

Click the files one by one (e.g. __init__.py, manifest.json, vacuum.py) and copy its contents to files with the same name within a folder named exactly like the platform. In this case: roomba/__init__.py, roomba/manifest.json, etc. If there is a translations folder, you can copy only the languages that are relevant to you.

TIP: Clicking “RAW” and then cmd+S or cntrl+S may make this process easier as you can either copy or simply download the files with the right name.

Once you have a folder with all the files in the component, see next section on what to do with it.

Uploading as a custom_components

Regardless of what method you use to download it, you should have now a folder called like your platform. In our case, roomba.

Simply, add the platform folder to your custom_components on your system. Your component is now downgraded to support YAML.

However, before restarting, keep reading as you may need to update the configuration.

Make Changes to the Configuration

You have just downgraded your component to an older version. It is possible that the configuration.yaml requires to be downgraded as well. How to know this?

The documentation for the components lives within a different component called home-assistant/home-assistant.io. Access it and navigate to source/_integrations/ and click the file with the platform name you want (e.g. roomba.md).

What you are seeing is the current configuration of the component. What we want now is to revert to the point in history where we downgraded our component. For this, we will go again to History and find the change before our config_flow.py was added. In our case, it was April 11. Click on [<>] button, to revert the component to that point in history.

Once again, you will be taken to the root directory. Navigate again to source/_integrations and open the platform.md file that you want.

Scroll down to the Configuration section and compare.

TIP: Click in RAW to read it clearly.

For example, in the case of roomba, I had to change from:

# Example configuration.yaml entry
roomba:
  - host: IP_ADDRESS_OR_HOSTNAME
    blid: BLID
    password: PASSWORD

to:

# Example configuration.yaml entry
vacuum:
  - platform: roomba
    host: IP_ADDRESS_OR_HOSTNAME
    username: BLID
    password: PASSWORD

The change from platform (e.g. roomba) to generic device (vacuum: - platform: roomba) may be common; but do note that there might be other changes too. For example, here blid was the new name for the former username and had to revert it.

Delete existing integration

Once you have downgraded the component by adding it to custom_components and updated the configuration, your YAML configuration and component are ready.

Before restarting, you should remove the existing configuration to avoid conflicts. Navigate to Configuration > Integrations. Find the integration you want click the 3 vertical dots and click Delete.

Restart core system

Once the old (new?) UI integration is deleted. You can restart your system. Your YAML configuration should be working now.

(Optional) Upgrade system

Once you have migrated all the components that you want, you can freely upgrade your home-assistant and your components should still be working as YAML. If new components were migrated to YAML, you may need to repeat this process for all your components.

(Optional) Manual maintenance

Once you do this, your components will be reverted to old version. Most of them will work as is, however, some may require or could benefit from some maintenance. This is especially true if you want to keep upgrading Home-Assistant.

The most common maintenance that I encountered is migrating from Device to Entity. For example, in the case of roomba I had to change:

from homeassistant.components.vacuum import VacuumDevice

class RoombaVacuum(VacuumDevice):

to:

from homeassistant.components.vacuum import VacuumEntity

class RoombaVacuum(VacuumEntity):

Other optional improvements could involve migrating from setup_platform to async_setup_platform to improve system performance.

Now, you are in control of what happens with your components


Risks

  1. In general, having old and outdated components is not a recommendable thing to do. However, compared to not updating your whole system, not updating certain components is a lesser security issue and might be preferable.

  2. This solution is based on the fact that (1) custom_components load before core components and (2) setup_platform and other YAML methods are not meant to be deprecated as per this blog post. If this changes in the future, there will be no way to have YAML components at all.

  3. As you are reverting to an older point of history, your components will no longer update and receive improvements or broken changes. For example, imagine that roomba changes its API endpoints. This will completely break the component. (However, this is also true if you’re not updating your system). If you want the best of YAML Configuration, while having updates, you will need to create the custom_component yourself. You can read an example on this PS4 experiment.


Thank you!

PS: Copy of this post in Github repository as an experiment, with showcase of code.

9 Likes

Thanks for such a well documented method of keeping YAML config alive!

4 Likes

For the record as well on my particular experience:

The components where I tried this: cast, harmony, roomba, spotify.

  • In the case of cast, I also manually removed the warning that was added recommending the UI set up over the YAML.

  • I also downgraded hue. However, I did not find any version where YAML was fully supported. If someone finds a good version, let me know. However, I downgraded to a copy on a point of history where YAML was still detected so it doesn’t get removed in the future. Not perfect, but better some support than none, I guess?

I was stuck on 0.109 and I was able to upgrade to 0.113.3 after doing this.

I moved all the Device to Entity, and had to change a few configurations as explained (e.g. cast to media_player, roomba to vacuum).

1 Like

Like you I was staying put on an earlier version but I was looking at digging in somewhere. After reading all the new release notes and considering how trouble free testing an update is when you are using a container version of core HA I decided to move ahead to 0.113.0 and have now stopped at 0.113.2 for now and have not had any impact from loss of YAML config for my list of integrations. I am finding the improvement in the database that has come with 0.113.x has been fantastic so thanks to those who have worked hard on that! In fact I have NEVER been able to use “Logbook” or “History” for more than a day or so after a new release until 0.113.x and in fact they load quite briskly on my QNAP NAS. Again congrats to all involved in that advance!

Haha…Maybe you could post this method as a PR to the HA YAML Documentation?

In any case this section is so far out of date and bordering on being utterly irrelevant right now:

Home Assistant uses the YAML syntax for configuration. YAML might take a while to get used to but is really powerful in allowing you to express complex configurations.

For integrations that you want to use in Home Assistant, you add code in your configuration.yaml file to specify its settings. This especially applies to integrations that are not yet available to configure through the UI.

I can’t contribute to non-owned repositories without adding my employer as an owner, which I do not really fancy. Also, I kinda was told that my proposals are not very well welcomed, so I do not see if it would get approved :slight_smile:

Nonetheless, I would be happy if someone else adds it there.

2 Likes