Gut Check: a weekly checkup for your Home Assistant install using TypeSafe AI's Jev model (HACS)

My Home Assistant install has about 1,300 entities, and on a typical day about 150 of them are unavailable. Most of those are harmless, like a laptop that’s asleep or leftovers from something I removed. The few that need fixing get lost in the pile. An integration that fails to start is worse, because it only shows an error in the integrations list, so I could miss it for weeks.

So I built Gut Check. It looks over your install once a week and puts anything that needs you in Settings > Repairs, where you act on it or ignore it.

What it checks

  1. Home health check: Sorts unavailable entities into expected, worth fixing, and safe to remove. Only the worth-fixing ones get a Repairs card, and the card clears itself when the entity comes back.
  2. Update review: Reads the release notes on pending updates and flags the ones that might break something (renamed or removed features, manual steps). It never installs anything.
  3. Area suggestions: Suggests one of your existing areas for each device that has none. You confirm or decline each card.
  4. Device class suggestions: Suggests a device class for sensors that report a unit but have none, so they get the right icon, unit conversion, and a spot on the Energy dashboard.
  5. Stuck integration check: Sorts integrations that failed to start into passing glitch, sign-in problem, and broken for good.
  6. Critical label suggestions: Finds smoke, CO, gas and leak sensors and shutoff valves you haven’t marked critical yet.
  7. Diagnostic sensor suggestions: Suggests hiding sensors about the device itself (Wi-Fi signal, SIM carrier, phone storage) that clutter auto-generated dashboards.
  8. Recorder suggestions: Counts which entities write to the recorder most and suggests which ones you could exclude, with a snippet to paste into configuration.yaml yourself.

1-5 are on by default. The other 3 can be enabled in the integration’s Configure flow.

How it decides

The judgment calls go to Jev, a decision model from TypeSafe AI. Gut Check sends a short description of each item and a fixed list of answers, and Jev picks one and says how likely it is to be right. It can’t write free text, invent a new option, or tell Home Assistant to do anything. When it isn’t confident, Gut Check marks the item unsure and leaves it alone.

Prereqs

A free TypeSafe AI API key is required. There’s no add-on or local model to run. On my install, one run of every check costs about a cent. You set a daily token budget in Gut Check (500,000 tokens is the default – plenty of overhead), and Gut Check refuses any run that would go over it. Gut Check exposes 2 sensors that show tokens used and cost today.

In building this integration (running the checks multiple times, using about 3.2MM tokens total), I’ve used $0.114 of the $5.00 free balance I started with when I created my Jev account. For a typical user running every check weekly on an install like mine (~1,300 entities) cost $0.009 each run (9/10 of a cent). :slight_smile:

Install

Add https://github.com/funkadelic/ha-gutcheck to HACS as a custom repository (category: Integration), install it, then add Gut Check from Settings > Devices & services and paste your TypeSafe AI API key.

Open your Home Assistant instance and open a repository inside the Home Assistant Community Store.

If it sorts something wrong on your install, please open a GitHub issue and I can take a look! Loved to hear any feedback on how it worked out for your install.

1 Like

This looks really interesting. Great idea.

Nice idea. One case I don’t think it would catch, from my own install. The power sensor of my LocalThings fridge got no updates for 4.5 days, but it never went unavailable, it just kept showing 66 W. The log had an error from LocalThings a few minutes after the last update, then no warning or error from it until I reloaded it. If I read the README right, the health check only looks at unavailable entities. What gave it away for me was the last update time, so a sensor that normally changes over 1,200 times a day but hasn’t for days might be worth a card too.