Thanks but I already spend waaay too much time here and the discord channel sounds like an even deeper time hole.
I’ll muddle through this somehow …
I just asked a very similar question to baloob in the new release thread. Maybe hang out there and see what the answer is? At least there you may have his attention. 
Saw the answer. Didn’t help me. Would that it were so simple! 
- Just pulled docker container 0.89.0.
-
custom_components/mqttcontains MQTT’s__init__.pyplus my customizedclimate.pyandalarm_control_panel.py(which worked with 0.88.1). - 0.89 fails to load my customized versions. The warning messages suggest it’s still using the stock versions of
climate.pyandalarm_control_panel.py
I stopped the docker container, purged all files in custom_component/mqtt/__pycache__, restarted the container and … no joy; it still chooses to use the stock versions.
FWIW, I know it’s using the stock versions because the warning messages complain about unexpected keys (which only the customized versions knows how to handle) and, of course, the behavior of the two platforms lack the enhancements I put into the customized ones.
Meh, I’ll try again tomorrow (… like copy the entire MQTT directory structure into custom_components)
He hasn’t answered my follow on question yet. Maybe there will be an answer in there.
Some progress to report but it’s pretty clear people will experience headaches. I know because I’m living it right now.
I copied the entire stock MQTT component directory (and its sub-directory for light) to custom_components/mqtt. For good measure, I also purged the contents of the __pycache__ sub-directory.
Now 0.89 finds my customized alarm_control_panel.py (and loads it successfully) as well as the non-customized components I use such as light, switch, sensor, binary_sensor, and cover. Progress!
However, there’s still no joy in Hassville because 0.89 stumbles with my customized climate.py. It rejects it because it is unable to resolve STATE_AUTO.
Well now, that’s a head-scratcher because STATE_AUTO is not something I added to my customized climate.py. It also exists in the 0.89 version of climate.py.
Here’s the error message:
HASS[1457]: 2019-03-07 09:05:52 ERROR (MainThread) [homeassistant.loader] Error loading custom_components.mqtt.climate. Make sure all dependencies are installed
HASS[1457]: Traceback (most recent call last):
HASS[1457]: File "/usr/src/app/homeassistant/loader.py", line 166, in _load_file
HASS[1457]: module = importlib.import_module(path)
HASS[1457]: File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
HASS[1457]: return _bootstrap._gcd_import(name[level:], package, level)
HASS[1457]: File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
HASS[1457]: File "<frozen importlib._bootstrap>", line 983, in _find_and_load
HASS[1457]: File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
HASS[1457]: File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
HASS[1457]: File "<frozen importlib._bootstrap_external>", line 728, in exec_module
HASS[1457]: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
HASS[1457]: File "/config/custom_components/mqtt/climate.py", line 16, in <module>
HASS[1457]: from homeassistant.components.climate import (
HASS[1457]: ImportError: cannot import name 'STATE_AUTO' from 'homeassistant.components.climate' (/usr/src/app/homeassistant/components/climate/__init__.py)
HASS[1457]: 2019-03-07 09:05:52 ERROR (MainThread) [homeassistant.loader] Unable to find platform mqtt. Search path was limited to path of component: custom_components
HASS[1457]: 2019-03-07 09:05:52 INFO (MainThread) [homeassistant.setup] Setting up climate
HASS[1457]: 2019-03-07 09:05:52 INFO (MainThread) [homeassistant.setup] Setup of domain climate took 0.0 seconds.
STATE_AUTO is defined outside the realm of the MQTT component. It’s defined in the Climate component, in homeassistant/components/climate/const.py.
Do I also need to copy that file into custom_components/climate/?
Based on what I’m experiencing, I’d say the breezy, easy-peasy documentation for this Breaking Change assumes a ‘best-case’ scenario.
I found the source of the problem and the solution does not involve copying homeassistant/components/climate/const.py.
The 0.89 version of MQTT’s climate.py contains a subtle difference from the 0.88.1 version of the same file.
Here’s how 0.88.1 gets STATE_AUTO:
from homeassistant.components.climate import (
ATTR_OPERATION_MODE, DEFAULT_MAX_TEMP, DEFAULT_MIN_TEMP,
PLATFORM_SCHEMA as CLIMATE_PLATFORM_SCHEMA, STATE_AUTO, STATE_COOL,
Here’s how 0.89 gets STATE_AUTO:
from homeassistant.components.climate.const import (
ATTR_OPERATION_MODE, DEFAULT_MAX_TEMP, DEFAULT_MIN_TEMP,
STATE_AUTO, STATE_COOL,
0.88: homeassistant.components.climate
0.89: homeassistant.components.climate.const
Just enough to break my customized climate.py which is based 0.88 source code.
I updated my custom version and now 0.89 loads it successfully (and, most importantly, it works).
So the solution is … troubling. Maybe I misunderstood the reasoning behind this architectural change (no more partial custom component overlays) but I thought one factor was to ensure the customized component’s integrity in the face of underlying changes.
Today, if you want, you can override the Hue light platform, but not the other parts of the Hue integration. If a future update evolves the Hue component, removing or changing internal methods that the custom platform relied upon, the custom platform will start failing (like this report).
To avoid this, we’re going to no longer allow custom components to be partial overlays
Yet this was an example of a custom component requiring additional modifications in order to adapt to underlying changes. It failed it the same way when I had attempted to use a customized version of climate.py based on 0.80 in version 0.86. There were so many changes between 0.86 and 0.80 that it was understandable that my customized component would fail.
To be clear, I’m not finger-pointing but simply wish to understand how this architectural change didn’t insulate my customized climate.py from an underlying modification.
Your custom component is outdated. As you can see in the 0.89.0 release, the STATE_AUTO constant is imported from homeassistant.components.climate.const: https://github.com/home-assistant/home-assistant/blob/0.89.0/homeassistant/components/mqtt/climate.py#L14-L16
Yes, thank you, I discovered that was the case. The same was true, several weeks ago, when I attempted to use a customized MQTT climate.py based on 0.80 code in version 0.86. Naturally, it failed. There were so many changes in the MQTT component from 0.80 to 0.86 that it was understandable that my climate.py would fail.
I believed this latest architectural change (no more partial overlays) was a strategy to insulate my ‘partial overlay’ from underlying changes. However, I find I still have to modify it to adapt to the new way climate.py resolves constants (between 0.88 and 0.89). Effectively, I’m still having to adapt the code for new versions (as in the past) plus there’s the extra task of copying the entire family of MQTT components.
Same juice but involving a lot more squeeze!
I realize this change is ostensibly to provides benefits but, honestly, my first contact with it isn’t revealing them to me. Perhaps I’ve completely misunderstood its application or simply found a use-case where it offers no self-evident advantages.
Anyway, all’s well that ends well and I’ll adapt to the new way of doing things.
i also have a question about new custom component, i am still using the harmony as a custom component
so i have a new file structure : custom_components\harmony
in here;
- remote.py
- init .py
but i receve 2 warnings?
2019-03-07 15:17:17 WARNING (MainThread) [homeassistant.loader] You are using a custom component for harmony which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you do experience issues with Home Assistant.
2019-03-07 15:17:17 WARNING (MainThread) [homeassistant.loader] You are using a custom component for harmony.remote which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you do experience issues with Home Assistant.
why do i have 2 warnings instead of 1 ?
If the component is sourced from custom-components, 0.89 will issue a warning. One warning for each component.
Here’s what I mean:
I have all MQTT components in custom_components/mqtt.
Two of them are customized: climate.py and alarm_control_panel.py.
All others are not customized.
I also use light, switch, cover, sensor, binary_sensor, lock.
When Home Assistant starts it issues warnings that I’m using custom components for climate, alarm_control_panel, and for light, switch, cover, sensor, binary_sensor and lock. So I get at least 8 warnings.
Here’s an abbreviated version of the log:
2019-03-07 10:01:39 WARNING (MainThread) [homeassistant.loader] You are using a custom component for mqtt.alarm_control_panel which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you do experience issues with Home Assistant.
2019-03-07 10:01:40 WARNING (MainThread) [homeassistant.loader] You are using a custom component for mqtt.lock which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you do experience issues with Home Assistant.
2019-03-07 10:01:40 WARNING (MainThread) [homeassistant.loader] You are using a custom component for mqtt.binary_sensor which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you do experience issues with Home Assistant.
2019-03-07 10:01:40 WARNING (MainThread) [homeassistant.loader] You are using a custom component for mqtt.climate which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you do experience issues with Home Assistant.
2019-03-07 10:01:40 WARNING (MainThread) [homeassistant.loader] You are using a custom component for mqtt.cover which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you do experience issues with Home Assistant.
2019-03-07 10:01:40 WARNING (MainThread) [homeassistant.loader] You are using a custom component for mqtt.sensor which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you do experience issues with Home Assistant.
2019-03-07 10:01:40 WARNING (MainThread) [homeassistant.loader] You are using a custom component for mqtt.switch which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you do experience issues with Home Assistant.
2019-03-07 10:01:40 WARNING (MainThread) [homeassistant.loader] You are using a custom component for mqtt.light which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you do experience issues with Home Assistant
Sure, you are not 100% insulated from all breaking changes, but it’s a lot better. Changes as we did to media player here, are one time clean ups which are now possible now that platforms are no longer in that folder. We’re breaking up a 1000 line file into smaller, more manageable ones.
It appears the excitement isn’t over. I just discovered another surprise.
I created a simple automation whose trigger uses platform: mqtt.
- id: 'light_toggle'
alias: Light Toggle
trigger:
- platform: mqtt
topic: "test/toggle"
action:
- service: light.toggle
entity_id: light.kitchen
It doesn’t work because 0.89 reports it cannot find the mqtt.automation platform.
HASS[1457]: 2019-03-07 23:35:23 ERROR (MainThread) [homeassistant.loader] Unable to find platform mqtt. Search path was limited to path of component: custom_components
HASS[1457]: 2019-03-07 23:35:23 ERROR (MainThread) [homeassistant.setup] Unable to prepare setup for platform mqtt.automation: Platform not found.
It lives here /components/automation/mqtt.py but 0.89 can’t find it because now it assumes everything related to MQTT is in custom_components.
This is … unexpected. Will I have to copy that into custom_components as well?
oh crap!!! This is exactly what @pnbruckner found with his sun.py as well and he fixed it… there was quite a discussion here about this. I didn’t have any issue and didn’t even do the rename as I wasn’t getting a warning… but @klogg and I think Phil had the same issue with an automation…
Yes. I simply created custom_components/sun/automation.py and put this inside:
from homeassistant.components.automation.sun import *
1 Like
Thanks, I’ll try that.
IMHO, this whole process of customizing one or two MQTT components has gone from straightforward to Byzantine.
BTW, I did the same thing with other files I didn’t need to customize with my custom amcrest component. Just create a file in the right place with the right name and put an appropriate from xxx import * statement in it for each of the files you don’t need to customize. It’s a trade-off. Either you copy the file, and you’re “insulated” from changes (unless you’re not, like you found), or you do an import of the standard module and you get updates as they come and you only deal with incompatibilities if and when they come.
1 Like
Thanks again! Tested and confirmed it works.
Created:
/custom_components/automation/mqtt.py
containing:
from homeassistant.components.automation.mqtt import *
In lieu of creating duplicates of standard modules in custom_components, this technique redirects Home Assistant to use the standard modules. In effect, this re-establishes the concept of ‘partial overlays’!
There’s the irony. It failed to deliver on one of its chief benefits: insulating a custom component from underlying changes. I understand it can’t be 100% insulation but, damn, this failed to deliver on its promise immediately. I believe the same thing happened to you with the armcrest component.
I was going to make a joke that this added complexity is just a roundabout way of nudging developers to submit their components into the main codebase (and not as custom components). However, in light of your past experiences with submitting your sun component, maybe it’s not all that funny.
1 Like
Wouldn’t just making your modified mqtt component it’s own stand alone custom_component be easier?
1 Like
How does one create customized versions of MQTT climate.py and alarm_control_panel.py that stand alone?
By ‘stand alone’ do you mean to create a new platform called my_mqtt and then place the modified climate.py file here:
custom_components/my_mqtt/climate.py
Then in the config file, one would refer to it like so:
climate:
- platform: my_mqtt
I have not tried it …
EDIT
Tried, it works, and is far easier than trying to make the modified MQTT components remain part of the MQTT platform.
I was a bit skeptical of this change when I first saw it. It just seems to add unnecessary hoops to jump through. The insulation from breaking changes is nice in theory, but just doesn’t work that well in practice.
I have a feeling most people will opt for just importing the standard components/platforms and bypass the restrictions on partial overlays once they find out about it. It’s just so much easier to not have to maintain copies of everything that you don’t actually need to customize. I’d rather just deal with the breaking changes in my one custom piece when they happen rather than having to recopy everything else with every release just to stay up to date on the stuff I didn’t customize.
It seems like this makes keeping current even more difficult than it was before. Instead of just watching for changes in my one custom piece and whatever dependencies it uses, I have to watch for changes in everything if I want them to stay up to date.