I’d like to propose a novelty feature that could significantly enhance our daily lives by helping us keep track of where we store our household items. This feature would be particularly beneficial for households where multiple people store and retrieve hundreds of items, as well as for individuals who have difficulty remembering where things are placed. Additionally, it could serve as a valuable tool for people with visual impairments, offering them an easier way to locate objects in their home.
Feature Overview
The proposed feature would allow Home Assistant to maintain a database of object locations based on voice commands. When storing an object, the user could inform the voice assistant of the object’s location. For instance, a user could say, “I’m placing the ebook in the second drawer” or “I’m putting the TV warranty in the top shelf.” The system would then store this information, along with the room where the command was issued, allowing the user or others to quickly find the object later by simply asking where it is.
Key Benefits
-
Simplifies Object Retrieval:
- No more searching through every drawer or cabinet. Simply ask Home Assistant where the object is, and it will tell you the last recorded location. Or suggest a posible location based on similar items.
-
Supports Multiple Users:
- Perfect for households where different members may store items. Anyone can retrieve information about where an object was last placed.
-
Accessible for Visually Impaired Users:
- This feature can make everyday life easier for those with visual impairments, enabling them to locate objects in their home without assistance.
-
Cost-Effective and Simple:
- No need to rely on NFC tags or Bluetooth trackers allowing thousands of items to be tracked in a natural way. (However, the feature could be made compatible with NFC, QR codes or Bluetooth systems in the future, offering even more flexibility for users who have those technologies already in place).
Software Changes Required
-
Database Schema Update:
- Introduce a new table to store object names, their associated locations, and the timestamp of when the object was stored. The schema would include fields such as
object_name,location_description,room_name,timestampand any other field such astagsthat may be needed further down the road.
- Introduce a new table to store object names, their associated locations, and the timestamp of when the object was stored. The schema would include fields such as
-
Creation of a Known Storage Locations Database:
- Create a database of known locations within the home (e.g., “top drawer,” “black cabinet,” “second shelf”) specific to each room. This will allow the assistant to understand and infer which location a user might be referring to, even if the user doesn’t provide all the details in their command. This would enable the Large Language Model to make educated guesses or suggestions based on partial information.
-
Voice Command Parsing:
Updating the intent recognition system to handle commands like “store object”, “locate object.” and “delete object”. This of course is a task much easier to handle by a LLM or a route LLM that could recognize the intent and route it to a specific set of instructions. -
Integration with Device Location:
- Modify the voice assistant module to capture and store the device’s location (room name) when the command is issued. This may require integrating with existing presence detection features or enhancing the location tracking capabilities. Not mandatory but it’s easier to announce where we store something if we don’t have to specify the room each time.
Potential for Future Enhancements
While the initial implementation could focus on simplicity, there are opportunities for future enhancements:
- Integration with NFC tags or QR codes for more precise tracking.
- Suggested storage locations based on patterns of use or existing storage habits. For instance if your locations list and associated categories is sent to a LLM, the assistant can suggest a location for a new item. “Where can I store this keyboard that I don’t use anymore?” (Of course he doesnt know if It would fit, but we can ask for additional places on that case)
- Follow up questions: Allow the assistant to ask clarifying questions when there is ambiguity (ex: the assistant is unsure whether an item being stored is a new item or an existing one).
- harder to implement but some items may contain other items, a box in a drawer can contain smaller items.
- Visual interface within the Home Assistant app to browse or search the objects location history.
Final Thoughts
I’m aware that it might take time to adapt to the habit of announcing where items are stored, and there’s a risk of items being moved without updating the system. However, the assistant could use patterns and location history to help suggest where to find these items if they’re not in their last known place.
While it will never be perfect, I still believe it can be extremely useful. You don’t need to track everything you own. Maybe you use it only for certain objects that you rarely use and might have difficulty finding later, or for items that don’t have an obvious location. The choice is yours.
Implementing this feature is no longer extremely difficult. Before the advent of LLM models, it would have been challenging, but now it’s within reach. If the Dev team isn’t interested, perhaps the community could develop a complement or a GitHub hacks tool.
I would love to hear feedback on this idea.
TL;DR
I’m proposing a new feature for Home Assistant to help track where household items are stored using voice commands. Users can announce where they’re placing items, and the system will remember the location, making it easier to find things later. Feedback from the community is welcome!
(Edited, added TL;DR)