Needle 2 Agentic LLM

Development
,
1

Anyone seen this? Seems like a match made in heaven for hass on-device voice assistant usage. (But granted, you still need a STT model.)

Project: GitHub - cactus-compute/needle: 14MB foundation model for tiny devices; phones, wearables, smart home, and robots. · GitHub
Demo: Needle 2 - The 14 MB Agentic LLM for Tiny Devices | Cactus

2

Hey @Glaciero - Roman here from Cactus.

Thank you for sharing! Needle 2 was specifically trained on Home Assistant-style tasks, but we’re working on improving it even further.

If you have any feedback on the model or improvement suggestions, we’d be glad to incorporate it in the next version.

3

I mucked around with this and cobbled together needle running on a M1 Mac mini and using the openal extended conversations HACS install…

Its like SUPER quick, basically instant.

image
image1148×1094 42.9 KB

4

How did you get it to integrate with HA? OpenAI integration with a custom url pointing to it?

5

Sure!

  • Shim: needle_shim.py on the Mac Mini translates OpenAI-style requests into Needle CLI calls, one subprocess per request. Runs as a LaunchDaemon.
  • HA side: HACS’s extended_openai_conversation (not core, which can’t point at a custom base_url), plus a manual openai version patch to its manifest.
  • Tools: per-entity functions didn’t disambiguate “the light” well, so redesigned around the kitchen’s group light entity instead of the two individual fixtures. (was using the kitchen lights as a test)
6

Hi, I created a addon/app for needle2