A generative agent integration based on langchain and langgraph

lindos · March 17, 2025, 1:12pm

The home-generative-agent project uses LangChain and LangGraph to create a generative AI agent that interacts with and automates tasks within a HA smart home environment. The agent understands your home’s context, learns your preferences, and interacts with you and your home to accomplish activities you find valuable. Key features include creating automations, analyzing images, and managing home states using various LLMs (Large Language Models). The architecture involves both cloud-based and edge-based models for optimal performance and cost-effectiveness. Installation instructions, configuration details, and information on the project’s architecture and the different models used are included. The project is open-source and welcomes contributions.

Cao_Hoa · March 22, 2025, 4:03am

hi i like your project very much.
i installed in hass. when i go to Integrations
installing Home Generative Agent got this error Config flow could not be loaded: {“message”:“Invalid handler specified”}
please help me have a nice day

alexandrechoske · March 24, 2025, 1:42am

Hi, I would love to test it but got stuck on pgvector install hahaha
Need docker?
How to suppose do this installation?

kombihacke · March 30, 2025, 2:45pm

Nice project! Unfortunetly i have some problems setting this up, see here:

github.com/goruck/home-generative-agent

Error: Config flow could not be loaded: 500 Internal Server Error Server got itself in trouble

opened 01:15PM - 01 Feb 25 UTC

Neowonder

documentation

### System Health details ## System Information version | core-2025.1.4 -- | -…- installation_type | Home Assistant OS dev | false hassio | true docker | true user | root virtualenv | false python_version | 3.13.1 os_name | Linux os_version | 6.6.62-haos-raspi arch | aarch64 timezone | Europe/Prague config_dir | /config <details><summary>Home Assistant Community Store</summary> GitHub API | ok -- | -- GitHub Content | ok GitHub Web | ok HACS Data | ok GitHub API Calls Remaining | 5000 Installed Version | 2.0.5 Stage | running Available Repositories | 1511 Downloaded Repositories | 13 </details> <details><summary>AccuWeather</summary> can_reach_server | ok -- | -- remaining_requests | 16 </details> <details><summary>Home Assistant Cloud</summary> logged_in | true -- | -- subscription_expiration | February 17, 2025 at 1:00 AM relayer_connected | true relayer_region | eu-central-1 remote_enabled | true remote_connected | true alexa_enabled | true google_enabled | true cloud_ice_servers_enabled | true remote_server | eu-central-1-3.ui.nabu.casa certificate_status | ready instance_id | 0098ce260922407f81f0044e8eec6ced can_reach_cert_server | ok can_reach_cloud_auth | ok can_reach_cloud | ok </details> <details><summary>Home Assistant Supervisor</summary> host_os | Home Assistant OS 14.2 -- | -- update_channel | stable supervisor_version | supervisor-2024.12.3 agent_version | 1.6.0 docker_version | 27.2.0 disk_total | 57.8 GB disk_used | 25.1 GB healthy | true supported | true host_connectivity | true supervisor_connectivity | true ntp_synchronized | true virtualization | board | rpi5-64 supervisor_api | ok version_api | ok installed_addons | Node-RED (19.0.0), Whisper (2.4.0), Piper (1.5.2), Samba share (12.4.0), Mosquitto broker (6.5.0), Home Assistant Google Drive Backup (0.112.1), Advanced SSH & Web Terminal (20.0.0), openWakeWord (1.10.0), File editor (5.8.0), MariaDB (2.7.2), ESPHome Device Builder (2024.12.4), Matter Server (7.0.0), Assist Microphone (1.3.0) </details> <details><summary>Dashboards</summary> dashboards | 15 -- | -- resources | 2 views | 73 mode | storage </details> <details><summary>Recorder</summary> oldest_recorder_run | January 21, 2025 at 12:54 PM -- | -- current_recorder_run | February 1, 2025 at 2:04 PM database_engine | mysql database_version | 10.11.6 </details> <details><summary>Sonoff</summary> version | 3.8.2 (c4b6fda) -- | -- cloud_online | 7 / 8 local_online | 6 / 6 </details> ### Checklist - [x] I have enabled debug logging for my installation. - [x] I have filled out the issue template to the best of my ability. - [x] This issue only contains 1 issue (if you have multiple issues, open one issue for each issue). - [x] This issue is not a duplicate issue of any [previous issues](https://github.com/ludeeus/integration_blueprint/issues?q=is%3Aissue+label%3A%22Bug%22+).. ### Describe the issue When trying to add the integration I get the following error message: Error Config flow could not be loaded: 500 Internal Server Error Server got itself in trouble ### Reproduction steps 1.I followed all the installation steps in the guide on the main GitHub integration page (btw, there is an error in your last step (10). "llama-3.2-vision-11b" should be "llama3.2-vision:11b" instead). 2.I put all the necessary files in the correct folders according to the instructions provided. 3.After restarting HA and trying to add the integration I get the abovementioned error message. ... ### Debug logs ```text Logger: aiohttp.server Source: /usr/local/lib/python3.13/site-packages/aiohttp/web_protocol.py:451 First occurred: 2:07:00 PM (1 occurrences) Last logged: 2:07:00 PM Error handling request Traceback (most recent call last): File "/usr/local/lib/python3.13/site-packages/aiohttp/web_protocol.py", line 480, in _handle_request resp = await request_handler(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.13/site-packages/aiohttp/web_app.py", line 569, in _handle return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.13/site-packages/aiohttp/web_middlewares.py", line 117, in impl return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 92, in security_filter_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 83, in forwarded_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 26, in request_context_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 86, in ban_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 242, in auth_middleware return await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 32, in headers_middleware response = await handler(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/http.py", line 73, in handle result = await handler(request, **request.match_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/decorators.py", line 81, in with_admin return await func(self, request, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 74, in wrapper return await method(view, request, data, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 172, in post return await self._post_impl(request, data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 179, in _post_impl return await super()._post_impl(request, data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 84, in _post_impl result = await self._flow_mgr.async_init( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... ) ^ File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1315, in async_init flow, result = await self._async_init(flow_id, handler, context, data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1340, in _async_init flow = await self.async_create_flow(handler, context=context, data=data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1536, in async_create_flow handler = await _async_get_flow_handler( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ self.hass, handler_key, self._hass_config ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/usr/src/homeassistant/homeassistant/config_entries.py", line 3333, in _async_get_flow_handler await _load_integration(hass, domain, hass_config) File "/usr/src/homeassistant/homeassistant/config_entries.py", line 3310, in _load_integration await async_process_deps_reqs(hass, hass_config, integration) File "/usr/src/homeassistant/homeassistant/setup.py", line 575, in async_process_deps_reqs await requirements.async_get_integration_with_requirements( hass, integration.domain ) File "/usr/src/homeassistant/homeassistant/requirements.py", line 54, in async_get_integration_with_requirements return await manager.async_get_integration_with_requirements(domain) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/requirements.py", line 170, in async_get_integration_with_requirements await future File "/usr/src/homeassistant/homeassistant/requirements.py", line 159, in async_get_integration_with_requirements await self._async_process_integration(integration, done) File "/usr/src/homeassistant/homeassistant/requirements.py", line 182, in _async_process_integration await self.async_process_requirements( integration.domain, integration.requirements ) File "/usr/src/homeassistant/homeassistant/requirements.py", line 264, in async_process_requirements self._raise_for_failed_requirements(name, missing) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/requirements.py", line 292, in _raise_for_failed_requirements raise RequirementsNotFound(integration, [req]) homeassistant.requirements.RequirementsNotFound: Requirements for home_generative_agent not found: ['langchain-openai==0.2.13']. ``` ### Diagnostics dump _No response_

Any ideas?

Best regards,
Chris

Cao_Hoa · May 8, 2025, 7:32am

I also received the same error as you. Anyone can help me.

lindos · May 9, 2025, 2:43am

Hello,

There seems to be a couple issues being reported here, one related to missing requirements in the manifest and the other running with docker or HAOS.

I updated the manifest.json some time ago. This should solve the former.

I have not tested this project with docker or HAOS, only with HA Core in a manual environment under Raspberry PI OS.

Happy to elaborate or help more if I can.

lindos · May 9, 2025, 2:47am

Hello, what problem did you find installing pgvector? I have not tested this project under docker, only with HA Core.