Sorry for going quiet, the thread grew faster than I could keep up with. I also didn’t want to answer @boheme61 with a guess, so I went to read the code first.
Short version: he’s right, it’s a bug, and it’s mine.
If your config has an include: block anywhere, the recorder runs in allowlist mode, and the order HA applies is this one:
include.entities -> record
exclude.entities -> skip
include.entity_globs -> record
exclude.entity_globs -> skip
include.domains -> record
anything else -> skip
The toggles in the panel only ever write to exclude.entities. That is the second line, sitting above your entity_globs on the third. So the moment you switch entities off (and “Deselect all” makes it much worse) those entries land on top of your own globs and kill them. Nothing in the UI tells you that is happening. JC, that is why your globs stopped recording. Sorry for the wasted evening.
It fails the other way round too: switching an entity on only removes it from exclude.entities. If nothing in your include block matches it, the last rule still skips it. Green toggle, no recording. Same thing @phantomsphorever reported in #15.
And the part I like least: since 0.8.32 the migration copies your old include.domains / include.entity_globs into the Manual exclusions field by itself. So migrating drops you straight into the mode where the panel is wrong.
For now, if you need one specific entity recorded, a glob with no wildcards matches only itself:
include:
entity_globs:
- sensor.your_entity_id
And if you don’t actually need allowlist behaviour, deleting the include block and staying in plain exclude mode is the saner option today. That is basically what @Frank_Beetz suggested, and it’s why it worked for him.
What I’m doing about it: first a warning in the panel when an include block is active, so at least you can see which entities no include rule covers. Then the real fix, toggles that write to include.entities when you are already in allowlist mode. Only when it’s already active. I’m not going to invent an include block for people who don’t have one, that would break everybody else. No date, because I’d miss it.
@NathanCu, on importing the YAML as it is: fair enough, and redoing 26k entities by hand is not something I would do either. Since 0.8.32 the import does bring exclude.domains, exclude.entity_globs, exclude.event_types, include.domains and include.entity_globs across, verbatim. The one it can’t represent is include.entities, which is the same hole as above. So it’s one fix and not two. If your config is include based I would honestly wait for that release before touching anything.
Anyone about to migrate, please update to 0.8.37 first. 0.8.36 and 0.8.37 fixed two migration bugs that are worse than this one: a custom db_url being dropped, so HA quietly started an empty database, and steps 2 and 3 being clickable without step 1, which could overwrite your recorder config with the defaults. And take a backup, I mean it.
On new entities defaulting to excluded, @boheme61, @CaptTom and @Deedee all asked for it here, so I’ve reopened #7. It also comes almost for free with the allowlist work, since that last rule already skips whatever isn’t matched by anything.
@CaptTom, about the size: yes, that number is .db + .db-wal + .db-shm added together. I did it that way because it’s what the disk really holds, but with a big WAL it’s misleading, so I’ll split it.
@Frank_Beetz, @Thyraz, @Deedee, thanks for the patience and for helping each other out here while I wasn’t around. And thanks in particular to @boheme61, because that report looked like a misconfiguration at first and turned out to be a real bug I would not have found on my own.