0.89 Breaking Change: Prevent partial custom component overlays

Configuration
21

Wouldn’t just making your modified mqtt component it’s own stand alone custom_component be easier?

1 Like
22

How does one create customized versions of MQTT climate.py and alarm_control_panel.py that stand alone?

By ‘stand alone’ do you mean to create a new platform called my_mqtt and then place the modified climate.py file here:

custom_components/my_mqtt/climate.py

Then in the config file, one would refer to it like so:

climate:
 - platform: my_mqtt

I have not tried it …

EDIT

Tried, it works, and is far easier than trying to make the modified MQTT components remain part of the MQTT platform.

23

I was a bit skeptical of this change when I first saw it. It just seems to add unnecessary hoops to jump through. The insulation from breaking changes is nice in theory, but just doesn’t work that well in practice.

I have a feeling most people will opt for just importing the standard components/platforms and bypass the restrictions on partial overlays once they find out about it. It’s just so much easier to not have to maintain copies of everything that you don’t actually need to customize. I’d rather just deal with the breaking changes in my one custom piece when they happen rather than having to recopy everything else with every release just to stay up to date on the stuff I didn’t customize.

It seems like this makes keeping current even more difficult than it was before. Instead of just watching for changes in my one custom piece and whatever dependencies it uses, I have to watch for changes in everything if I want them to stay up to date.

24

I’ve had a similar thought. :wink: I can understand what I assume is the ultimate goal, which would be to minimize disruption caused by poorly written and/or poorly maintained custom components. And I can even, kind of, understand implementing a system to help avoid that disruption. But honestly, the implementation seems a bit half-baked in that there were a lot more “gotchas” than maybe was expected. And I also think the implementers of that system may not be fully understanding the magnitude of all the extra work and frustration it has caused for both creators and users of custom components. I suspect they’re only seeing the “tip of the iceberg.” Which is kind of a shame, because one of the great benefits of this system is the ability for it to be fairly easily extended. Making that process more difficult can have the reverse effect – i.e., not nudging people towards contributing to the standard product, but rather throwing in the towel and going elsewhere.

Well, hopefully this is all temporary and the dust will settle soon and the result will be a better system overall.

Yes, it was pretty frustrating to do all that work to try and enhance the standard sun component (getting it to work, creating extra tests, updating documentation, etc., etc.), only for it to be rejected because it didn’t fulfill someone’s dream of what the sun component should be some day. But I haven’t been deterred from still trying – I’m in the process of submitting several incremental improvements to the standard amcrest component. :slight_smile:

3 Likes
25

maybe a little bit off topic. May i ask what changes you did and for what devices they are? And do you know if they broke some other mqtt stuff?
If not i’d suggest to “fix” the official component and have this solved once and for all (and others)

26

It’s in the very first thread post: climate.py and alarm_control_panel.py. The actual enhancements aren’t relevant to this discussion. However, here’s a thread for Enhanced version of MQTT HVAC which was a result of an earlier thread.

And do you know if they broke some other mqtt stuff?

They didn’t break anything, the Breaking Change did it.

If not i’d suggest to “fix” the official component and have this solved once and for all (and others)

Have you seen the submission process? It’s a steep hill to climb for hobbyists. The ability to tweak a standard module for one’s own purposes (and to easily share it with others), without the submission process, is an enormous advantage.

It used to be very easy to install/share this kind of customized component (copy the modified file into the correct folder and done). Now, “very easy” is not something I could comfortably use to describe the process. Plus the purported advantage of this change failed a real-world test.

2 Likes
27

I have a custom HVAC mqtt climate component too and it failed with 0.89 too, how to migrate it? I read all the breaking change and notes but still not know how.
it show: “Unable to find platform mitsubishi_mqtt. Search path was limited to path of component: custom_components”.

28

Is your customized climate.py file located here?

/custom_components/mitsubishi_mqtt/climate.py
29

Yes, but it show the error. I also change the import code from: from “from homeassistant.components.climate” to "import homeassistant.components.climate.const import "

30

It work now after change the import code :).
So don’t need to make extra step like you do above.

31

At first I didn’t understand what you meant … and then it all became clear. Why struggle with maintaining my customized components as being part of the mqtt platform when I can simply say they’re part of a completely separate platform. They may have originated from the mqtt platform but they’re not obliged to remain part of it.

Thank you, kind stranger, this is now my preferred solution to simplify life in light of this Breaking Change.

What I did:

I copied my modified MQTT climate.py and alarm_control_panel.py to:

custom_components\my_mqtt\

Then I revised climate and alarm_control_panel in configuration.yaml to use:
platform: my_mqtt
instead of:
platform: mqtt

Like this:

alarm_control_panel:
  - platform: my_mqtt
    name: Security System
    # etc ...

climate:
 - platform: my_mqtt
   name: Thermostat
   # etc ...

I restarted Home Assistant and, ta-dah, they work. Easy-peasy. Only warning messages are the expected ones indicating I’m using custom components for climate and alarm_control_panel.

This is, by far, much easier than trying to make the modified components remain part of the existing mqtt platform.

OK, maybe not quite ‘ta-dah’ because climate.thermostat had a unique_id therefore its revised version (configured to use a different platform: my_mqtt) becomes registered as climate.thermostat_2.

My Lovelace UI was configured to display climate.thermostat but could no longer find it. It’s an easy fix . I stopped Home Assistant, edited the relevant entries in .storage/core.entity_registry, and all was well after restarting Home Assistant.

2 Likes
32

Yes, this is what I was referring to.

33

Good that this works, but I don’t see it as being a real (long-term) solution to the problem. There are some obvious side effects and will it even work for all components/platforms? Plus, is there any advantage to either of these solutions over just using partial overlays?

It seems like partial overlays are still possible but just require more effort to set up than before. However, that effort is still less than what it takes to do them the “correct” way and override the entire component.

Besides all that, wasn’t it always possible to “isolate” your custom platform by overriding the entire component or making a new component before it was even mandatory? It’s basically just gone from “opt-in” to “opt-out” where everyone who wasn’t originally opted in now has to go through this process of opting out.

34

“Long term” in Home Assistant is … about 2 weeks.

is there any advantage to either of these solutions over just using partial overlays?

As of 0.89, partial overlays are gone. We can debate its merits as an intellectual exercise but it’s unlikely Paulus will resurrect them.

It seems like partial overlays are still possible but just require more effort to set up than before.

A simulation of partial overlays is possible using the redirection technique Phil described. Indeed it is more effort than when partial overlays were permitted. You need to create one redirection file for each component that is to remain ‘stock’.

So, in my case, where I only modified 2 MQTT components, I’d need to create at least 8 redirection files, one for each component I did not modify (light, switch, cover, lock, sensor, binary_sensor, device_tracker, etc) plus the MQTT automation. Yeah, I’d call that “more effort” than the way partial overlays worked.

wasn’t it always possible to “isolate” your custom platform by overriding the entire component or making a new component before it was even mandatory?

I guess the short answer is “possible but not necessary”.

35

Why wouldn’t this be a long-term solution? It conforms to the standard the devs set forth, and as 123 has confirmed it works. If you want to mod the whole way a specific bit of HomeAssistant works, you might as well submit a PR for merging. If you’re only modifying for specific use cases, a custom component would seem the best route to go, not overwriting the core bits.

36

Because I think the real “solution” shouldn’t include all of this extra stuff to workaround the limitations that are put in place. It is a solution to the current problem, but I just think the current problem shouldn’t exist or should have a better process in place for dealing with it.

Anyway, that’s all outside the scope of making custom components work with the current limitations, so I guess there’s not much reason to discuss it here. For now, either a new component or import the built-in components is the workaround for simulating partial overlays.

37

I would be remiss not to give credit where it’s due, specifically for the superior way the “Great Migration” organizes platforms and components. Kudos to Paulus and all who helped to re-organize it.

Let’s say we have a platform called SuperWidget. Prior to 0.88, modified versions of SuperWidget’s climate and alarm_control_panel components would be arranged like this:

custom_components/climate/superwidget.py
custom_components/alarm_control_panel/superwidget.py

From 0.88 onwards, they are arranged like this:

custom_components/superwidget/climate.py
custom_components/superwidget/alarm_control_panel.py

That’s neater. All of a platform’s components reside in a single directory. It makes it easier to create, and maintain, a derivative platform (like my_mqtt).

custom_components/my_mqtt/climate.py
custom_components/my_mqtt/alarm_control_panel.py

Thanks to all involved for this improvement!

38

so your climate.py and alarm_control_panel.py don’t use any other parts of the platform? Or if they did they would just use the mqtt standard stuff?
and the alternative is to do a redirect for all py files in the platform? are they listed somewhere?
I don’t have any custom components I have written but I’d like to understand the process…

I count 16 py files here https://github.com/home-assistant/home-assistant/tree/dev/homeassistant/components/mqtt and there is also a light folder with 3 more

39

That’s right, they use other parts of the stock MQTT platform. The file containing MQTT constants (const.py) is one that comes to mind. Keep in mind that my climate.py and alarm_control_panel.py are just tweaked versions of the stock versions so they will continue to have the same dependencies.

and the alternative is to do a redirect for all py files in the platform? are they listed somewhere?
The main suggestion is to copy over all related files. As you’ve discovered for the case of mqtt it means at least 16 files. So even if only tweaked two of them, if my configuration file refers to any unmodified components (like light, switch, sensor, etc) they all have to be in the custom_component/mqtt with the modified ones.

Phil’s suggestion is to use placeholder/redirection files that redirect Home Assistant to use the release version. For example, custom_components/mqtt/lock.py can contain from homeassistant.components.mqtt.lock import * to make Home Assistant refer back to the release version. I used this technique when I discovered I would also need to duplicate mqtt.automation.

The beauty of creating a derivative platform, like my_mqtt, is its simplicity. I don’t need to copy over any other files because Home Assistant is handling it as a unique platform, unrelated to the mqtt platform.

1 Like
40

I understood that bit… so if your own custom platform uses MQTT it will just use the stock HA platform? If so that’s pretty cool and would seem by far the best/simplest way to handle a custom component that modifies a built-in one.