OpenAI Conversation doesn't work with iOS Shortcuts

Mobile Apps Home Assistant Companion for Apple
1

I was able to get iOS Shortcuts to respond to simple automation commands using the default conversation agent through Siri on my iPhone. I was then able to successfully installed the OpenAI Conversation integration and get it working within Lovelace.

The Shortcuts stop working properly when the OpenAI Conversation integration is enabled, however. No matter what the input is, Siri responds with ‘OK.’ or ‘That’s done.’ to requests like ‘What’s the average depth of Lake Tahoe’. The integration via the Lovelace interface properly answers that question. Both use OpenAI credits so I know the requests are hitting their API.

Logs imply that the Assist function of the Hass iOS app is dropping the responses.

Working diag via Lovelace [ignore the confidently hallucinated GPT responses]:

Answer the users questions about the world truthfully.

If the user wants to control a device, reject the request and suggest using the Home Assistant app.

Now finish this conversation:

Smart home: How can I assist?
Mike: who made you?
Smart home: I am a smart home controlled by Home Assistant.
Mike: what is the time?
Smart home: The time is currently 2:45 PM.
Mike: what is the date?
Smart home: Today is Tuesday, April 28th.
Mike: when was the last president a republican?
Smart home: The last Republican President was George W. Bush, who served from 2001-2009.
Mike: who was the last president with a last name that starts with s?
Smart home: 
2023-02-25 15:29:23.139 DEBUG (MainThread) [homeassistant.components.openai_conversation] Response {
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "text": " The last president with a last name starting with 'S' was George H.W. Bush."
    }
  ],
  "created": 1677367762,
  "id": "cmpl-6nyGgxRLLhHSWcisnRgngJNslI1Vk",
  "model": "text-davinci-003",
  "object": "text_completion",
  "usage": {
    "completion_tokens": 20,
    "prompt_tokens": 1384,
    "total_tokens": 1404
  }
}

Failed diag via iOS Shortcut:

Answer the users questions about the world truthfully.

If the user wants to control a device, reject the request and suggest using the Home Assistant app.

Now finish this conversation:

Smart home: How can I assist?
Mike: Who is the last president with the last name that starts with S
Smart home: 
2023-02-25 15:30:09.044 DEBUG (MainThread) [homeassistant.components.openai_conversation] Response {
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "text": " The last president with a last name starting with S was George H. W. Bush."
    }
  ],
  "created": 1677367808,
  "id": "cmpl-6nyHQJgXRabaVLsoctwDQuqgzR6pF",
  "model": "text-davinci-003",
  "object": "text_completion",
  "usage": {
    "completion_tokens": 18,
    "prompt_tokens": 1289,
    "total_tokens": 1307
  }
}
1 Like