So I might have been wrong about not being able to improve that.
Although there are likely some challenges with making sure libyaml is available, Significantly improve yaml load times when the C loader is available by bdraco · Pull Request #73337 · home-assistant/core · GitHub showed it might be possible to make it ~9x-10x faster.
can we test drive that? sounds good: 10X faster…
AFAICT there is no 2022.6.6 yet. Did you mean 2022.6.5, the release notes indicate a speedup in that version. 2022.6: Gaining new insights! - Home Assistant
You are correct, my bad. Made the edit above
Is there a way to have multiple mqtt: and sensor: sections in a package YAML file? I have a lot of stuff that is grouped but this new format makes it much harder.
For example, I have a modem signal YAML file that brings in all the downstream MQTT sensors, then has min_max sensors using them all; then I bring in all the upstream MQTT sensors, then has min_max sensors using them all.
Most of my stuff flows like that…temp sensors, weather station, etc. where I grouped the sensors by the thing they sense (bring it in, filter it, max/min it, repeat) not by the platform they come in.
I tried to do this but after changing the MQTT ones to their own heading now I’m getting a bunch of errors…
2022-06-11 19:29:17 WARNING (SyncWorker_0) [homeassistant.util.yaml.loader] YAML file /config/packages_yaml/network_monitoring/arris_modem_signal.yaml contains duplicate key "mqtt". Check lines 64 and 1160
2022-06-11 19:29:17 WARNING (SyncWorker_0) [homeassistant.util.yaml.loader] YAML file /config/packages_yaml/network_monitoring/arris_modem_signal.yaml contains duplicate key "sensor". Check lines 703 and 1191
Yes. Its called packages. You can group config from any number of integrations into a logical package.
Right, I’ve been using packages - so I have 1 YAML package file for each type of thing.
In my modem YAML I have it define sensor, then I have blocks where for each related thing I define the MQTT sensor, min_max, etc. type (repeat for downstream, repeat for upstream).
Similarly I have a temp & humidity YAML file, which then defines the MQTT sensors, templates, filters, min_max, average (repeat by area, repeat all that for humidity portion of the same sensors)
I really don’t want to have to do dozens of separate YAML files for every individual sensor…but it sounds like that may now be the only option since MQTT is now unique needing its own separate sensor-heading instead of being under “sensor” like every other platform?
Unless the entire package is in one file then no you won’t be able to merge sensors using the legacy config style (under sensor with a platform key that has the integration id) with the new style (everything under the integration id). I’m not sure which integrations you’re referring to that are still using the legacy config style but you should expect them to migrate over time to the new style or to the UI.
With Mqtt you might want to consider simply using Mqtt discovery though as opposed to yaml config. This post above and the replies talk about the massive (read: 10x or more) performance improvement achieved by doing that and how to do that with an HA script.
I don’t think discovery works with the rtl2mqtt addon, but even if it did I’m not sure I want all the many random sensors in range or passing by suddenly dumped into HA automatically…surely that would create performance problems if every 433 and 315 MHz device in range was being imported. I couldn’t figure out a way to make it auto-discover the cable modem channel stats published to MQTT either.
I don’t think anything else is using a “new” style?
Templates the “new” way is missing features like friendly-name which creates problems when you have same display-name with different kinds of information (e.g. sensor.dryer_watts vs sensor.dryer_status are both just “dryer” for the name, but you can’t have duplicate names if that’s now the key) so I can’t use that…and it makes the friendly name unfriendly.
As far as I can tell, MQTT is the only thing throwing alerts that its changing. Nothing I’ve seen about filter, max_min, statistics all go under “sensor”.
I really hope they don’t force using the UI for everything, that’s horribly clunky especially when you have large numbers of repeating things or want to duplicate and do a simple find/replace. I’d much rather tweak stuff in an IDE and a few keystrokes than having to use the mouse with 500 clicks. I don’t understand why they want to eliminate a convenient way of configuring stuff. Plus you can easily do version control on YAML files, you can’t do version control and change management on a GUI form.
I do agree, but that ship sailed some time ago.
Wouldn’t a few hundred extra sensors create a lot of extra system load if they aren’t used?
There’s a LOT of stuff that passes by in range from remotes to TPMS sensors that pass into MQTT but aren’t mine. Also still have to map the numbers into useful names (and change them when you replace a sensor, or some when you replace batteries)
It does, everything does. It’s just an alternative to yaml config. Also:
That’s not how Mqtt discovery works? The way Mqtt discovery works is something sends a message under this topic:
<discovery_prefix>/<component>/[<node_id>/]<object_id>/config
Where discovery_prefix is typically homeassistant unless you have changed it. The message looks exactly like what you have in yaml right now except converted to json and you can optionally create devices to group entities (only supported in Mqtt discovery not yaml config).
I’m not familiar with the addon you’re using but if it’s not specifically doing Mqtt discovery then it’s not going to create entities out of random meters. It doesnt matter what messages it sends for the state of those meters, discovery messages are under an entirely different topic. But you can fire discovery messages yourself, it’s simply an alternative to yaml config.
Also if you are using an addon for rtl that doesn’t do Mqtt discovery for you you might want to find an alternative to simplify your setup. There’s a bunch of addons that do that if you search the forum. Hell I even maintain one, you’re more then welcome to use it.
I an the first to admit to knowing bugger all about rtl2mqtt, just what I saw by googling. But can’t you restrict it to certain protocols, but that might not narrow it enough.
Yeah, so it publishes like a bunch of stuff like (picking one example) homeassistant/sensor/rtl433/Acurite-Tower/2858 or homeassistant/sensor/rtl433/Hyundai-VDO/5011e85e which has that device’s fields.
Then that has to be mapped to something useful like “front porch temperature” (and humidity, and battery), or “this car, front right tire pressure” (and temperature, and battery)
I’ve not seen autodiscover work and ended up deciding it was easier to pull in the sensors I want manually, grouped in their appropriate package-file (along with the templates, filters, etc that have to go along to handle odd values and stats).
If you’re sating that every device would need to have the startup script for the addon modified, that sems even more insane since it breaks the concepts of encapsulation where files should contain like related information and be portable. The RF to MQTT initialization should be able to run on any machine without modification, hence the whole point of having configuration variables to begin with…and then the packages would have configuration in the YAML related to each package (which doesn’t need to care about where the sensor came in, just that it is manipulating it).
Home assistant mqtt discovery is usually done by the third party device you are connecting to home assistant. There is a list of 3rd party tools that use discovery here MQTT Discovery - Home Assistant.
The innovation that @CentralCommand was referring to was using a script that “fakes” what the 3rd party tool does by using mqtt.publish to load a discovery message to home assistant.
That method will still require manual setup as you’ll have to set up a script for each device or entity. OTOH if your third party tool does discovery, it will all be handled for you.
Sure, just put the script(s) in the package(s) too. It’s just as portable as all the other config HA understands.
I think there’s a monthly (?) automatic purge that should cause that to stabilize after roughly a month. Unless you’ve been adding entities or automations that are tracked and cause entries, something does look wrong.
I was originally going to post after 2022.6 that my database size was growing (after slowly reducing after 2022.5) but it wasn’t constructive so I held off for the time being. After growing from 900 to 950 MB over the last few weeks, it just suddenly dumped to around 700MB. Maybe try running a database purge manually? Other than that I can only think of perusing the logbook for what’s adding all the entries or manually navigating the SQL database for insight. You should be able to find something around the gaps up?
Bu default the database is purged every morning and 10 days data are kept.
I also read about a database maintenance and repack task which runs on the 2nd Sunday of every month. I can confirm that this happened for me today (Version 2022.5.5) and it shrunk the database by about 1/3.
I can’t wait to see the database improvements in 2022.6 when I finally get a chance to upgrade!