I use LocalAI for all my LLM needs, and wanted the ability to see (from HA obviously) what was installed, and various model details.
So, using the built-in LocalAI API I’ve built an integration that does just that: loryanstrant/HA-LocalAI-Monitor: An integration for Home Assistant that provides information about your LocalAI instance.
Apart from just stats, the model attributes are also shown:
The same goes for backends:
It also has the ability to shut down models, which I use as part of my automations to save NVRAM when I know the model won’t be needed again immediately (yes, I have enabled LocalAI to shut models down after an idle period, but this is faster).
Combined with my GPU model monitor and using the table card, I can do this:
