The future of YAML

Hi @petro

I am also trying to help those users who may get confused by this forced YAML migration; or with some of the statements that you posted in reply, that at least me (and I think @finity) do not find fully accurate or at least clear.

Some of us do find that debate is a healthy way to contribute and clarify, as opposed to sparing smart statements and leave.

In any case, you’re not interested in debating; you made that clear. We can park it here. I can continue debating and discussing with whomever finds this conversation relevant.

Thank you!

4 Likes

As was I.

And that’s what I thought I was doing…trying to help clarify a point of confusion. Maybe not on your part but possibly on someone else’s part that might come by later and misread your comments.

2 Likes

Wow! we really have to stop cross posting like this. It’s getting really creepy.

1 Like

Can someone expand on this statement? Or at least provide a little more clarity. is this referring to the configuration.yaml file itself, or configuration in general?

Any integration polling data from devices like a hub (Hue) or a cloud based service (SmartThings) will only be allowed to be configured via the GUI.

1 Like

This is a solution (as owners said, do it yourself) for those of you who, like me, stopped updating Home Assistant due to YAML Components. I wanted to share a way to downgrade your components to the versions where YAML was being used, so you can upgrade the rest of the platform.


Introduction

Git (GitHub) keeps version control of all changes to components and you can revert to any version. On this process, we will simply download a copy of the home-assistant/core component at a certain point of history, where YAML was still supported.

As such, the key point is that YAML has to have been supported at some point for this to work.


Process

Before you start, it is advisable that you create a backup (a.k.a snapshot) and download it in case something goes wrong and you need to revert to this point in history.

Finding the right version

The first task is to find the most recent version where YAML was still supported. For this, we are going to go to home-assistant/core > homeassistant > components. Then select the platform that you want to downgrade. For this demo, we will use roomba.

If UI is the main configuration, there will be a file called “config_flow.py”, click it and navigate to History on top right.

Scroll all the way down and check the date of the first commit. If there are many changes, you many need to click “Older” a few times. In this case, it’s April 11, 2020 and the commit is Add config_flow for Roomba (#33302). This is the commit where config_flow.py (UI) was created. We probably want to revert to the version exactly before this one.

Within that view, click now on the name of the platform folder, to go to the History page for the whole component. Your objective now is to find the commit before where config_flow.py was added. Again, you may need to click “Older” a few times. Memorizing the date before will help you in doing this faster.

In this case, we want to revert the component to “Add prettier (in pre-commit and CI) (#33693)”. The last version before UI was added.

Downloading the component

Via Command Line

If you are familiar with git (GitHub), you can clone the repository at that version (example):

git clone https://github.com/home-assistant/core.git
cd core
git checkout 39336d3ea367e5c50e88fa09165257290a07b150

Where 39336d3ea367e5c50e88fa09165257290a07b150 is the commit id. You can get it by clicking on the commit.

Once done, copy the platform folder (e…g homeassistant/components/roomba). See next section to know what to do with it. Once you have copied the folder we need, you can delete the overall folder containing the other files of the home-assistant/core repository.

Manually

If you are not familiar with git or do not want to clone the repository, there’s also a manual way:

Once you have identified the version you want, click the button [<>] to revert the view to that commit/point in history.

Once you click it, you will be back to the main home-assistant/core folder, but with a key difference. Now, you will be in the view of the repository in the point of history where you selected.

Navigate again to your component folder: home-assistant/core > homeassistant > components > (e.g. roomba)

Click the files one by one (e.g. __init__.py, manifest.json, vacuum.py) and copy its contents to files with the same name within a folder named exactly like the platform. In this case: roomba/__init__.py, roomba/manifest.json, etc. If there is a translations folder, you can copy only the languages that are relevant to you.

TIP: Clicking “RAW” and then cmd+S or cntrl+S may make this process easier as you can either copy or simply download the files with the right name.

Once you have a folder with all the files in the component, see next section on what to do with it.

Uploading as a custom_components

Regardless of what method you use to download it, you should have now a folder called like your platform. In our case, roomba.

Simply, add the platform folder to your custom_components on your system. Your component is now downgraded to support YAML.

However, before restarting, keep reading as you may need to update the configuration.

Make Changes to the Configuration

You have just downgraded your component to an older version. It is possible that the configuration.yaml requires to be downgraded as well. How to know this?

The documentation for the components lives within a different component called home-assistant/home-assistant.io. Access it and navigate to source/_integrations/ and click the file with the platform name you want (e.g. roomba.md).

What you are seeing is the current configuration of the component. What we want now is to revert to the point in history where we downgraded our component. For this, we will go again to History and find the change before our config_flow.py was added. In our case, it was April 11. Click on [<>] button, to revert the component to that point in history.

Once again, you will be taken to the root directory. Navigate again to source/_integrations and open the platform.md file that you want.

Scroll down to the Configuration section and compare.

TIP: Click in RAW to read it clearly.

For example, in the case of roomba, I had to change from:

# Example configuration.yaml entry
roomba:
  - host: IP_ADDRESS_OR_HOSTNAME
    blid: BLID
    password: PASSWORD

to:

# Example configuration.yaml entry
vacuum:
  - platform: roomba
    host: IP_ADDRESS_OR_HOSTNAME
    username: BLID
    password: PASSWORD

The change from platform (e.g. roomba) to generic device (vacuum: - platform: roomba) may be common; but do note that there might be other changes too. For example, here blid was the new name for the former username and had to revert it.

Delete existing integration

Once you have downgraded the component by adding it to custom_components and updated the configuration, your YAML configuration and component are ready.

Before restarting, you should remove the existing configuration to avoid conflicts. Navigate to Configuration > Integrations. Find the integration you want click the 3 vertical dots and click Delete.

Restart core system

Once the old (new?) UI integration is deleted. You can restart your system. Your YAML configuration should be working now.

(Optional) Upgrade system

Once you have migrated all the components that you want, you can freely upgrade your home-assistant and your components should still be working as YAML. If new components were migrated to YAML, you may need to repeat this process for all your components.

(Optional) Manual maintenance

Once you do this, your components will be reverted to old version. Most of them will work as is, however, some may require or could benefit from some maintenance. This is especially true if you want to keep upgrading Home-Assistant.

The most common maintenance that I encountered is migrating from Device to Entity. For example, in the case of roomba I had to change:

from homeassistant.components.vacuum import VacuumDevice

class RoombaVacuum(VacuumDevice):

to:

from homeassistant.components.vacuum import VacuumEntity

class RoombaVacuum(VacuumEntity):

Other optional improvements could involve migrating from setup_platform to async_setup_platform to improve system performance.

Now, you are in control of what happens with your components


Risks

  1. In general, having old and outdated components is not a recommendable thing to do. However, compared to not updating your whole system, not updating certain components is a lesser security issue and might be preferable.

  2. This solution is based on the fact that (1) custom_components load before core components and (2) setup_platform and other YAML methods are not meant to be deprecated as per this blog post. If this changes in the future, there will be no way to have YAML components at all.

  3. As you are reverting to an older point of history, your components will no longer update and receive improvements or broken changes. For example, imagine that roomba changes its API endpoints. This will completely break the component. (However, this is also true if you’re not updating your system). If you want the best of YAML Configuration, while having updates, you will need to create the custom_component yourself. You can read an example on this PS4 experiment.


Thank you!

PS: Copy of this post in Github repository as an experiment, with showcase of code.

9 Likes

Thanks for such a well documented method of keeping YAML config alive!

4 Likes

For the record as well on my particular experience:

The components where I tried this: cast, harmony, roomba, spotify.

  • In the case of cast, I also manually removed the warning that was added recommending the UI set up over the YAML.

  • I also downgraded hue. However, I did not find any version where YAML was fully supported. If someone finds a good version, let me know. However, I downgraded to a copy on a point of history where YAML was still detected so it doesn’t get removed in the future. Not perfect, but better some support than none, I guess?

I was stuck on 0.109 and I was able to upgrade to 0.113.3 after doing this.

I moved all the Device to Entity, and had to change a few configurations as explained (e.g. cast to media_player, roomba to vacuum).

1 Like

Like you I was staying put on an earlier version but I was looking at digging in somewhere. After reading all the new release notes and considering how trouble free testing an update is when you are using a container version of core HA I decided to move ahead to 0.113.0 and have now stopped at 0.113.2 for now and have not had any impact from loss of YAML config for my list of integrations. I am finding the improvement in the database that has come with 0.113.x has been fantastic so thanks to those who have worked hard on that! In fact I have NEVER been able to use “Logbook” or “History” for more than a day or so after a new release until 0.113.x and in fact they load quite briskly on my QNAP NAS. Again congrats to all involved in that advance!

Haha…Maybe you could post this method as a PR to the HA YAML Documentation?

In any case this section is so far out of date and bordering on being utterly irrelevant right now:

Home Assistant uses the YAML syntax for configuration. YAML might take a while to get used to but is really powerful in allowing you to express complex configurations.

For integrations that you want to use in Home Assistant, you add code in your configuration.yaml file to specify its settings. This especially applies to integrations that are not yet available to configure through the UI.

I can’t contribute to non-owned repositories without adding my employer as an owner, which I do not really fancy. Also, I kinda was told that my proposals are not very well welcomed, so I do not see if it would get approved :slight_smile:

Nonetheless, I would be happy if someone else adds it there.

2 Likes

I understand your situation. For my part I have looked at the process of editing the docs by adding a PR in the past to rectify errors or outdated rubbish and bailed out because it was simply too onerous a task.
And always with the possibility the PR will not be approved!

And as I type this I get this load of bilge shouting at me from the preview pane:

Encourage everyone to get involved in the conversation

You’ve replied 3 times to @nitobuendia in this particular topic!

A great discussion involves many voices and perspectives. Can you get anybody else involved?

And don’t forget, if you’d like to continue your conversation with this particular user at length outside of public view, send them a personal message.

Yeah right!

I’ve been able to submit documentation PRs and get them approved, it’s honestly not that bad :stuck_out_tongue:

3 Likes

Thats good for you then…keep up the good work!

Let’s say I want to start with a clean HA install, I don’t want to restore snapshots… Before with yaml, just overwrite the configuration.yaml file, and everything was setup… How to do it now with config flow?

So you want to partially restore a backup (the configuration.yaml part) but not the rest?

Indeed, just some integrations at once, like copy pasting some stuff from yaml file

I think you are out of luck.

But UI configs are generally quick and easy to set up.

Hmm, that’s too bad , was so easy before

1 Like

And has been discussed to death.