I added person and media_player but of both only 1 seems to be in. How can I see what pinecone contains?
EDIT: found it, it is scanning .storage/core.entity_registry which does not contain these.
EDIT2: source 1: me, source 2 unknown , (media_player_samsungtv is in the list of entities)…anyhow, digging further 
For me the openassist doesn’t work unfortunately 
Seems like pinecone won’t sync my entities. Anyone else experiencing this?
Nope, all fine…show logs/errors?
So, I have come a little further. I got some debugging going in my logs, and then I found complaining about openai key. Changed account and made a new api key, and pinecode were able to sync entities.
Now, I’m having some issues with mindsdb
This is the output of my logs:
2023-06-25 11:23:36.713 DEBUG (MainThread) [custom_components.openassist] Handling state change event for openassist_prompt
2023-06-25 11:23:36.713 DEBUG (MainThread) [custom_components.openassist] Generating embeddings for new state
2023-06-25 11:23:36.713 DEBUG (SyncWorker_26) [custom_components.openassist] Creating embedding
2023-06-25 11:23:37.173 DEBUG (MainThread) [custom_components.openassist] Embeddings generated, preparing payload for Pinecone
2023-06-25 11:23:37.173 DEBUG (SyncWorker_19) [custom_components.openassist] Sending GET request
2023-06-25 11:23:37.701 DEBUG (MainThread) [custom_components.openassist] Payload prepared, sending POST request to Pinecone
2023-06-25 11:23:37.701 DEBUG (SyncWorker_35) [custom_components.openassist] Sending POST request
2023-06-25 11:23:38.449 DEBUG (MainThread) [custom_components.openassist] POST request to Pinecone complete, processing response
2023-06-25 11:23:38.449 DEBUG (MainThread) [custom_components.openassist] All matches metadata: {"entity_id": "light.veranda_uplights", "original_name": "AK001-ZJ210 DC4F22A2A118", "platform": "flux_led"}, {"entity_id": "light.downlights_gang", "original_name": "Downlights Gang", "platform": "knx"}, {"entity_id": "light.downlights_2", "original_name": "Downlights", "platform": "loxone"}, {"entity_id": "light.downlights", "original_name": "Downlights", "platform": "loxone"}, {"entity_id": "light.downlights_ida", "original_name": "Downlights Ida", "platform": "knx"}
2023-06-25 11:23:38.449 DEBUG (MainThread) [custom_components.openassist] Firing OpenAssist update event
2023-06-25 11:23:38.449 DEBUG (MainThread) [custom_components.openassist] OpenAssist update event fired
2023-06-25 11:23:38.452 INFO (MainThread) [custom_components.openassist.sensor] OpenAssistSensor received update event
2023-06-25 11:23:38.452 INFO (MainThread) [custom_components.openassist.sensor] Event data: new_state=How many lights , metadata={"entity_id": "light.veranda_uplights", "original_name": "AK001-ZJ210 DC4F22A2A118", "platform": "flux_led"}, {"entity_id": "light.downlights_gang", "original_name": "Downlights Gang", "platform": "knx"}, {"entity_id": "light.downlights_2", "original_name": "Downlights", "platform": "loxone"}, {"entity_id": "light.downlights", "original_name": "Downlights", "platform": "loxone"}, {"entity_id": "light.downlights_ida", "original_name": "Downlights Ida", "platform": "knx"}
2023-06-25 11:23:38.452 INFO (MainThread) [custom_components.openassist.sensor] Prepared prompt for GPT-4: If the query is a generic question, respond with:
2023-06-25 11:23:38.452 INFO (MainThread) [custom_components.openassist.sensor] Getting GPT-4 response
2023-06-25 11:23:38.452 DEBUG (MainThread) [custom_components.openassist.sensor] Asking MindsDB
2023-06-25 11:23:38.880 INFO (MainThread) [custom_components.openassist.sensor] MindsDB response: {'title': 'Error', 'detail': 'There is no user record with id=None'}
2023-06-25 11:23:38.880 INFO (MainThread) [custom_components.openassist.sensor] GPT Response:
2023-06-25 11:23:38.880 INFO (MainThread) [custom_components.openassist.sensor] Executing service
2023-06-25 11:23:38.880 ERROR (MainThread) [custom_components.openassist.sensor] No response to execute
2023-06-25 11:23:38.882 INFO (MainThread) [custom_components.openassist.sensor] Sending notification
2023-06-25 11:23:38.882 ERROR (MainThread) [custom_components.openassist.sensor] No message to send
2023-06-25 11:23:38.883 INFO (MainThread) [custom_components.openassist.sensor] Updating Home Assistant state
2023-06-25 11:23:38.884 INFO (MainThread) [custom_components.openassist.sensor] Home Assistant state updated
So it seems to fail due to this error:
2023-06-25 11:23:38.880 INFO (MainThread) [custom_components.openassist.sensor] MindsDB response: {‘title’: ‘Error’, ‘detail’: ‘There is no user record with id=None’}
Possibly mindsdb issue, this is my return just now, have not touched it for about 1 week so not sure why API exceeded.
2023-06-25 13:07:21.048 INFO (MainThread) [custom_components.openassist.sensor] MindsDB response: {'message': 'MLEngineException: [openai/gpt4hass]: API key has exceeded its quota, please try 1) increasing it or 2) using another key.'}That will be an issue with your session cookie.
Answering to myself, my ‘free’ API alotment with openai is gone, can not use it anylonger and as it is tied to my phone#, I can not setup a new/free one.
I am not sure how many 'call’s one gets for free … I donot have the impression I used this a lot (Maybe 30-40 requests?) but paying for this is not (!) an option, too expensive for me
the OpenAI API used in this project is only for ADA Embedding model, which is extremely low compared to GPT3 or 4.
for example:
GPT4 8K context input = $0.03 / 1K tokens output = $0.06 / 1K tokens
compared to…
Ada training = $0.0004 / 1K tokens usage = $0.0016 / 1K tokens
To upload all my entity data to pinecone for example cost me around $0.05 then it was less then $0.001 per query/question afterwards.
OK, thanks and indeed nothing to worry about… I recently made a mistake with aviationstack and got a nbill of 45USD for 2 days which I used only for dev work.
** New Update **
We no longer need the MindsDB Session Cookie.
This has now been replaced with your MindsDB user Email and Password.
Your sensor in configuration.yaml should now look like this!
sensor:
- platform: openassist
your_name: ""
## New Section
mindsdb_model: ""
mindsdb_email: ""
## New Section
mindsdb_password: ""
notify_device: ""
Upgraded, works fine.
One thing I noticed is that the response to a simple text request is not json, this was fine before.
i.e. on “tell me a joke” I get this in return
MindsDB response: {'column_names': ['response'], 'context': {'db': 'mindsdb'}, 'data': [['{\n message: "Sure, here\'s a joke for you: Why don\'t scientists trust atoms? Because they make up everything!"\n}']], 'type': 'table'}
2023-07-02 08:34:54.180 INFO (MainThread) [custom_components.openassist.sensor] GPT Response:
{
message: "Sure, here's a joke for you: Why don't scientists trust atoms? Because they make up everything!"
}
2023-07-02 08:34:54.181 INFO (MainThread) [custom_components.openassist.sensor] Executing service
2023-07-02 08:34:54.181 ERROR (MainThread) [custom_components.openassist.sensor] Could not decode response as JSON
As message is not “message” is logically crashes.
However, when I request smt like ‘living room light on’ then the message is (!) with double quotes
EDIT: previous version
2023-07-02 08:43:05.909 INFO (MainThread) [custom_components.openassist.sensor] MindsDB response: [{'response': '{\n "message": "Sure, here\'s a joke for you, Arjan: Why don\'t scientists trust atoms? Because they make up everything!"\n}'}]
2023-07-02 08:43:05.910 INFO (MainThread) [custom_components.openassist.sensor] GPT Response:
{
"message": "Sure, here's a joke for you, Arjan: Why don't scientists trust atoms? Because they make up everything!"
}
def ask_mindsdb changed (as you know of course) but not sure why the message / “message” is different when you ask a simple question vs. one that returns actions on light
EDIT2: found the solution, the first line in the prompt needs a slight change ref. to json
If the query is a generic question, respond with a valid JSON object:
1 Like
Thanks for investigating. Pushed this change.
Next on my list (when I have a bit of time) is on the weather, I only get ‘none’ in reponse for my sensors or on ‘what is today’s weather?’ it wants to start a service weather.update. But it also does not find all my weather sensors, I have OpenW, Accu, MeteoFrance.
included_domains: "light, weather, switch, media_player, person, sensor"
log
2023-07-03 09:57:30.409 INFO (MainThread) [custom_components.openassist.sensor] MindsDB response: [{'response': '{\n "actions": [\n {\n "domain": "weather",\n "service": "update",\n "entity_id": "weather.forecast_home_hourly_3",\n "data": {}\n },\n {\n "domain": "weather",\n "service": "update",\n "entity_id": "weather.mougins",\n "data": {}\n },\n {\n "domain": "weather",\n "service": "update",\n "entity_id": "weather.forecast_home_hourly_2",\n "data": {}\n },\n {\n "domain": "weather",\n "service": "update",\n "entity_id": "weather.forecast_home_3",\n "data": {}\n },\n {\n "domain": "weather",\n "service": "update",\n "entity_id": "weather.forecast_home_2",\n "data": {}\n }\n ],\n "message": "The weather updates for all the requested locations have been fetched successfully."\n}'}]
2023-07-03 09:57:30.410 INFO (MainThread) [custom_components.openassist.sensor] GPT Response:
{
"actions": [
{
"domain": "weather",
"service": "update",
"entity_id": "weather.forecast_home_hourly_3",
"data": {}
},
{
"domain": "weather",
"service": "update",
"entity_id": "weather.mougins",
"data": {}
},
{
"domain": "weather",
"service": "update",
"entity_id": "weather.forecast_home_hourly_2",
"data": {}
},
{
"domain": "weather",
"service": "update",
"entity_id": "weather.forecast_home_3",
"data": {}
},
{
"domain": "weather",
"service": "update",
"entity_id": "weather.forecast_home_2",
"data": {}
}
],
"message": "The weather updates for all the requested locations have been fetched successfully."
}
2023-07-03 09:57:30.411 INFO (MainThread) [custom_components.openassist.sensor] Executing service
2023-07-03 09:57:30.412 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/config/custom_components/openassist/sensor.py", line 192, in _async_handle_update
await self.execute_service(self.hass, response)
File "/config/custom_components/openassist/sensor.py", line 233, in execute_service
await hass.services.async_call(domain, service, data)
File "/usr/src/homeassistant/homeassistant/core.py", line 1857, in async_call
raise ServiceNotFound(domain, service) from None
homeassistant.exceptions.ServiceNotFound: Unable to find service weather.updateHi, i have this error:
> Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/config/custom_components/openassist/__init__.py", line 109, in state_change_handler
xq = await hass.async_add_executor_job(create_embedding, new_state.state, MODEL)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/openassist/__init__.py", line 39, in create_embedding
return openai.Embedding.create(input=input, engine=model)['data'][0]['embedding']
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/openai/api_resources/embedding.py", line 33, in create
response = super().create(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
response, _, api_key = requestor.request(
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 226, in request
resp, got_stream = self._interpret_response(result, stream)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 619, in _interpret_response
self._interpret_response_line(
File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 679, in _interpret_response_line
raise self.handle_error_response(
openai.error.RateLimitError: You exceeded your current quota, please check your plan and billing details.
I have just installed the custom component and try to create Pinecode index, a paid account in Openai is needed for this?
Depending on the number … yes, I setup a paid account and have not been charged yet. As per above, it is very cheap but after you set it up, you should check the first few days to see if things are in line with expectations of being cheap
Hi! I’m having a problem with the response from mindsDB. Generic questions work fine, but trying to eg. turn off a light results in an error:
Question: turn off gaming room light
Data:
"
2023-09-23 17:25:31.234 INFO (MainThread) [custom_components.openassist.sensor] MindsDB response: {'column_names': ['response'], 'context': {'db': 'mindsdb'}, 'data': [['{\n domain: "light",\n service: "turn_off",\n entity_id: "light.gaming_room",\n data: {},\n message: "The gaming room light has been turned off."\n}']], 'type': 'table'}
2023-09-23 17:25:31.236 INFO (MainThread) [custom_components.openassist.sensor] GPT Response:
{
domain: "light",
service: "turn_off",
entity_id: "light.gaming_room",
data: {},
message: "The gaming room light has been turned off."
}
2023-09-23 17:25:31.236 INFO (MainThread) [custom_components.openassist.sensor] Executing service
2023-09-23 17:25:31.237 ERROR (MainThread) [custom_components.openassist.sensor] Could not decode response as JSON
2023-09-23 17:25:31.245 INFO (MainThread) [custom_components.openassist.sensor] Sending notification
2023-09-23 17:25:31.248 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/config/custom_components/openassist/sensor.py", line 239, in _async_handle_update
message = json.loads(response).get('message') if response else None
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
This seems to happen because the property names are not in quotes so the json decoder doesn’t work.
What if I told you guys, that there is LocalLLM HA Integration which allows to:
- generate Text using
text-generation-webui API - generate Images using
stable-siffusion API[WIP] - use any open-source LLM models
- use the
contextof your Home Assistant setup - talk natively just like with build-in Assist to detect & run Intents
- when no Intent is matched it generates response using LLM Model based on context
- exposes Options supported by external APIs
- exposes services to
generate_text,generate_image,generate_text_from_image,count_tokensand other to manage models in external APIs
…but it’s not yet published? Does that sound exiting?
4 Likes
I figured this one out, openassist did not recognize my command due to my own bad naming of light entities.
How are you using whisper to input a prompt to openassist? I have faster-whisper running locally, is there a way to use that as the input?
Is there any way to use this integration to replace the “assist” in home assistant? “Assist” can never understand my commands, but this OpenAssist integration works great (like 100X better, really). Right now I just use the text to speech on my phone to input the prompt, but I have 2 Willow ESP32-S3 Boxes that I would like to use to prompt “OpenAssist”