This isn’t a specific solution, but I wanted to show how I’m utilizing llama3 along with Apple/Android solutions, mostly in HA.
Sharing this because it might inspire someone.
I’ve integrated four apps:
- Dictation from an Apple Watch
- Dictation from an Android device (currently trying to use a remote from a Mi Box for this)
- An app on an old Android phone that receives calls from my phone number and SMS starting with a password and forwards them to the system.
- An app for inventory management and tracking of possessions (somewhat jokingly)
How it works:
- The transcription is sent to HA (Home Assistant) with llama3 8b
- llama3 8b classifies the user’s text into categories like notes, reminders, calendar, smart home controller
- Based on this, the user’s speech is sent to a specific agent who handles only one thing, e.g., runs an AppleScript (notes/calendar/reminders etc), saves inventory status to a database, controls HA.
I had a similar system built previously (built years ago) with Snowboy wake word and working as a voice assistant via live google transcription (STT) for home management.
But llama3 changed everything. Now I have one place where I can say what I want, how I want, and it does everything I want.
(Sorry for everything being in Polish, I’m doing this for my family, but I’ve added subtitles for those interested)
Example of running applescript, but it is also triggering HA services, etc.
The inventory app is an app that interprets my speech and assigns items so I can, for example, know how many ESP32s I have. It’s based on that joke: https://www.youtube.com/watch?v=iYQ6aq85EFg (polish with translations)
The inventory app was also integrated to HA, but I’ve found custom server is better.