Custom_components revived to support external addons

Feature Requests
1

Hi,

I’m playing around with Home Assistant, and one if its great features is how easy it is to extend :slight_smile: . Just place a file in the custom_components/ folder and you’re good to go. Many examples or showcases in this forum state “put the following components in your custom_components”. And that’s great!

While this works and leads to quick results, I don’t think it’s a good way to keep custom extensions managed. There will for sure always be components that are neither mature enough nor have a large enough target audience to be included in the offical source tree. Just like “I tinkered around to get my special needs satisfied. No comments, ugly code but works. I’ll share it if someone is interested”.

I just started with HA, but knowing me and my play instinct, the custom_components folder will for sure look cluttered in just a short amount of time. Not knowing what file originates where and what files belong together. Being a developer, this is a nightmare for me.

So here is my proposal: Let’s add support for 3rd party plugins. Directly to HA, not as a hass.io addon to target as many users as possible. The idea is to allow to specify pip packages that are resolved automatically and considered custom components. As pip also supports loading from archives, one can easily point to e.g. a github repository for this just as even stated in the dev docs.

So the configuration.yaml would contain entries like

addons:
  - name: foo
    # uploaded to PyPI
    install: "ha-addon-foo==1.2.3"
  - name: bar
    # from github
    install: "https://github.com/some-user/ha-addon-bar/archive/v0.0.5.zip#ha-addon-bar==0.0.5"
    # force update on start
    update: true

where both addons could provide either new components or add platforms to existing ones.

And those addon lines would exactly be would be referred to then in the forum posts. Would even support updates. And could be the entry point for an addon to become mature enough to be officially included in HA itself - or be used to allow for simple beta-testing of a new or updated component. If the whole idea got successful, one could even think about a custom addon repository.

As for the loader, I would propose to

  1. create an abstraction layer for component loading
  2. redefine the current behavior (custom_components and components folder)
  3. add the addon functionality as separate implemention. It would use the addons configuration and could e.g. use custom Decorators to define Components and Platforms.

We could therefore only use the custom_components folder as a playground. And for testing out modifications or development we could refer to outsourced packages. Even if it wasn’t for sharing, if you wanted to develop your own components, this would allow to keep them all separate in git repositories.

What do you think about this proposal?

2

Personally I think something like this is exactly what they should have aimed for in the first place instead of hassio and addons. Everyone on the same platform, on the same page, with the same solutions when things don’t go to plan.

1 Like
3

I don’t think you can really compare hassio addons to the addons that are talked about in this feature request. Here it’s about Python modules that are about adding support for new hardware etc… The addons for hassio are a copmletely different thing. They typically are full applications that are preconfigured to perform a useful task in conjuction with Home Assistant. Like the IDE addon to modify the configuration, SSH, DuckDNS, Let’s Encrypt. Some of these can’t even be replicated with Python, or would at least require a LOT of work to port from their original language to a pure Python implementation.
So in short: that’s a comparison of apples and pears.

@thewilli
There have been custom components which have evolved into fully integrated components / platforms in Home Assistant. So the custom components, to me, are already what you’re asking for. And it would have a drawback: some people would publish the extensions as custom components, others as addons. At what point should developers decide if their code is one or the other? Eventually addons would only end up being what custom components are right now in my opinion.

4

I was talking about the concept. No reason that

addon:
 - duckdns

Couldn’t be a way to do it, and it downloads what it needs and adds some options to the configuration tab to add your url and api key, or

addon:
  - configurator

And it adds an iframe with the IDE having downloaded what it needs.

As a concept that’s far better than making two ways of doing things that are both homeassistant, and yet completely different and confusing for the end user.

5

The big drawback of the current custom components is that everything is thrown together. That makes it impossible to properly maintain it without any hacks (yes, one could do some overlayfs mounts, …).

Just consider I wanted to add the foo platform to switch and light. From the forum I’d then copy the bar platform for the switch component. My diretory structure would look like:

├── light/
│   ├── foo.py
│   └── bar.py
└─── switch/
    └── foo.py

foo and bar don’t belong together but are mixed up. I can’t keep them separately in source control. And a few month later I don’t recall what file belongs to what concern. There is no easy way to remove all foo related items - or to just disable them or use a previous version for testing purposes. IMHO the custom_components is just too limited and adds too many drawbacks.

Well, I think this is especially a matter of documentation and presentation. If the documentation would discourage custom_components for everything except playgorund testing, and would allow to simply migrate custom_components to the newer format, it would help to be accepted very soon. Maybe even the deps folder could be used to start a new addon. And explaining why separation of concerns is a good thing should be possible, even to non-developers, don’t you think?

6

I might be misunderstanding something here. Let me further explain my point of view to see what I got wrong (using the configurator as an example).

The configurator addon is a single-file Python executable. On non hassio-installations it typically is just downloaded, marked executable, and auto-started with the help of systemd. An alternative would be the Docker-image, but that’s only works for people that have Docker installed. On hassio it’s also a Docker image (but specially prepared with some additional requirements), and doing it the manual way with simply downloading the file and starting by systemd is not possible. So given that there are multiple types of installations, what should adding the configurator internally do? Installation method 1 would only work on Installations with systemd, method 2 only when docker is available. And the hassio-variant of the Docker image of course won’t run without Docker, and with docker there would be other parts missing that the hassio-variant requires.

If all that

addon:
  - configurator

does would be installing the configurator addon in hassio, then it would be an alternative method to installing the addon via GUI. But that could not work on Hassbian or other types of systems. So I don’t see any benefit, especially since the roadmap aims at making the configuration as graphical as possible, ideally only confronting users with yaml when there’s no other way.

Addons like DuckDNS, nginx, MySQL, SSH, InfluxDB, Grafana etc. would even create more trouble since they contain compiled code. A Windows User (yes, there are people running HASS on Windows) couldn’t install any of those. Here only Docker would work. But again, not everybody is using Docker. And Docker shouldn’t be a requirement. On top of that, the setups of the users can be very different in how their network infrastructure looks like. This too has to be considered, and probably would cause a lot of trouble.

Hhmm, I guess I got carried away a bit while thinking about this. :smiley:

7

I think you are misunderstanding what I’m saying about the concept. I’ll gladly sit and chat my opinions (and how I think it could be implemented) with you over a pint, but it makes no difference to the diverging nature that this project seems determined to take, and is just distracting from the original purpose of the thread. My original comment was just meant as an approval of the idea, and my opinion on why I approved :slight_smile:

8

Since I don’t really use custom components: are there many that use more than one file? If so, it maybe should become a best practice to mention all related files at the top of the code files in a docstring. Somewhat like the files-section used in RPM files. If I read you correctly your concern is comparable to how much not-fun it is to remove all files a RPM or DEB has installed without being allowed to look what the package has done while installation.

Sure they belong together. They’re both subclasses of the light-platform.

You can’t do that with any platform in Home Assistant, no matter custom or native. Different platforms are spread out all across different folders. That’s just how Home Assistant organizies code.

If they belong together they all have the same name. find . -name whatever... would do the trick.

Custom components have to be enabled to be functional. Just remove their config and they are disabled.

Make the directory a git repo and use branches.

My guess is, that only a minority is using the custom_components folder for development, just because there aren’t really many users doing development. Those few that do should be able to organize themselves to keep track of where their own code can be found and what it does. And the majority of users simply would just have an additional folder of which they don’t know what it’s doing.

Now don’t get me wrong. Assuming there are custom components that consist of multiple files, publishing it via a simple URL could indeed make installation easier. But for single-file components, it’s really just a single copy & paste. So either I copy & paste an URL, restart, and all files get placed where they should be (not taking into account possible conflicts of different custom components), or I just copy & paste the raw code into a file of my choice, restart, and it works exactly the same way.

9

this prevents any kind of proper automation :roll_eyes:

They are not developed together and don’t relate to the same development and -lifecycle.

You for sure see the benefit from having a versioned repository dedicated to a single target system / concern / technology. Why not providing users a way to do so?
Modern development tools - may it be console- or gui based - provide great functionality in versioning, tracking changes, errors etc. If my product platform needed updates (and may it only be to increase the REQUIREMENTS field to work with a new Python library version, one needed to manually search through the complete folder structure. What if you forgot one file? Do you really want to search & replace over a directory tree containing unrelated items, or work on a manual todo list for this?

And you would do that all the time, and expect that from less technical users as well? Tough one for Windows users.

If I wanted to provide you with some custom compoments, that are WIP. Maybe I’m even waiting for PRs on GitHub. Then you can’t just clone that directory structore into your custom_components without overriding files already existent - or just have 2 of such repositories.

It might break down to one question, do you (not you personally but the community and devs) would like the idea of external addons, however the actual structure of those would look like?

10

As mentioned in your other thread, have a look at this issue: https://github.com/home-assistant/architecture/issues/124
It also mentiones metadata and and distribution of custom components. So maybe with that they’re already doing what you are looking for.

1 Like
11

As @danielperna mentioned, custom component developers should now be able to package their components in a single directory for distribution and placement in the custom components dir. Which means that you should now be able to git clone them into that directory and git pull to keep them updated.

12

Thanks for pointing that one out :+1:

Sure, and that’s a good thing! Still, my question is if there shouldn’t be a way to have an external ecosystem of components that are targetting less common components (do you really want to have literally everything in the base code tree) or just less mature and stable than the integrated ones. A way to distribute and update components from a source tree. It doesn’t necessarily need to have anything to do with custom_components.

13

I agree and i do think that’s the way things are moving - a core of well written and vetted components maintained by the Hass devs and a system for installing other components by others.