[Custom Component] extended_openai_conversation: Let's control entities via ChatGPT

I’ve had trouble with setting light colours and things like that. I’ve found a spec for this, now is more reliable

- spec:
    name: set_light_color
    description: Sets a color value for a light entity. Only call this function
      when the user explicitly gives a color, and not warm, cold or cool.
    parameters:
      type: object
      properties:
        color:
          type: string
          description: The color to set
        entity_id:
          type: string
          description: The light entity_id retrieved from available devices. 
            It must start with the light domain, followed by dot character.
      required:
      - color
      - entity_id
  function:
    type: script
    sequence:
    - service: light.turn_on
      data:
        color_name: '{{color}}'
      target:
        entity_id: '{{entity_id}}'

- spec:
    name: set_light_brightness
    description: Sets a brightness value for a light entity. Only call this
      function when the user explicitly gives you a percentage value.
    parameters:
      type: object
      properties:
        brightness:
          type: string
          description: The brightness percentage to set.
        entity_id:
          type: string
          description: The light entity_id retrieved from available devices. 
            It must start with the light domain, followed by dot character.
      required:
      - brightness
      - entity_id
  function:
    type: script
    sequence:
    - service: light.turn_on
      data:
        brightness_pct: '{{brightness}}'
      target:
        entity_id: '{{entity_id}}'

- spec:
    name: set_light_warm
    description: Sets a light entity to its warmest temperature.
    parameters:
      type: object
      properties:
        entity_id:
          type: string
          description: The light entity_id retrieved from available devices. 
            It must start with the light domain, followed by dot character.
      required:
      - entity_id
  function:
    type: script
    sequence:
    - service: light.turn_on
      data:
        kelvin: '{{state_attr(entity_id, "min_color_temp_kelvin")}}'
      target:
        entity_id: '{{entity_id}}'

- spec:
    name: set_light_cold
    description: Sets a light entity to its coldest or coolest temperature,
      only call this function when user explicitly asks for cold or cool temperature of the light.
    parameters:
      type: object
      properties:
        entity_id:
          type: string
          description: The light entity_id retrieved from available devices. 
            It must start with the light domain, followed by dot character.
      required:
      - entity_id
  function:
    type: script
    sequence:
    - service: light.turn_on
      data:
        kelvin: '{{state_attr(entity_id, "max_color_temp_kelvin")}}'
      target:
        entity_id: '{{entity_id}}'

Thank you! This looks promising. But could you edit your post and paste it as plain text/code? yaml indentation can be a b**ch and despite a few attempts I can’t seem to load the functions despite editing them to be in the right yaml format.

Done :smiley:
Btw i’ve found this somewhere, can’t remeber where…

Do you guys have any other links to share with new spec ideas?

I was hoping this thread would be a throve of ideas on how to make out HA work better with AI… :smiling_face_with_tear:

Why can’t I make my Extended AI tel me what the weather will be like tomorrow? He can say today because I told him how to respond if asked about todays’ weather. I also told him if I asked about upcoming days then he should use the get_current_weather spec. But I don’t think he understood that.
What’s the trick?

I set up mine using agents using this link and everything works fine so far.

Have you found a solution to this?

Anyone have this working with the new LLM Music Assistant script? voice-support/llm-script-blueprint at main · music-assistant/voice-support · GitHub

I added in my prompt to use this script when playing music, which it does, but it doesn’t pass the player or area to play on.
image

Hey @jekalmin i love so much your component. I know that there must be differences but is not possible to have somthing like this integration with gemini ai? You know, it’s basically free for home use… i wish to use function calling and so on with gemini but i don’t know how to do

As of the time of writing, what can this custom integration do that the standard one cannot ?

I’m wondering whether or not I need to use this, but the standard one seems to be able to read and control entities just fine ?

This one can connect to other OpenAI compatible servers on the Internet or locally, whereas the built-in integration only connects to OpenAI servers.

1 Like

But i found a major issue in the extended one … it doesn’t work well with coloring lights or thier brightness even after i put some of the functions above for those … however the original one works well doing that

I tried the music assistant script but it didn’t work for me. I told the integration to use the script when I ask to “play music” but it didn’t work.
I went back to just using spotify for music.

Ex: If I say “play some Prince”, then it plays some Prince music from the spotify integration on the sonos kitchen speaker.

That works but it always only plays one song.

Anyone would know how to play more than one song?

I have it play a lot of playlists too. I say plus the Jazz playlist and it’ll play just one song.

I told it to keep on playing down the playlist but it didn’t understand that.

Did you guys try it?

Can anyone explain how to determine the right value for Maximum Function Calls Per Conversation?

Also, I can’t find documentation of what “use tools” means.

Hi,
I’m using this component and loving it, but since a month or so it has lost “memory”. Any command requiring confirmation won’t be performed, because
AI doesn’t remember what I’m confirming. As a simple test I ask it to repeat last message but it anwers that it’s not programmed to. If I switch to openAI component joined with the same open AI account, everything is OK. I can conversate with AI and it remembers context. I prefer Extended openAI because of scripting capabilitties so I need help with restoring its “memory” :slight_smile: Forgive me my poor english. Thanks in advance for any help.
EDIT:
I’ve just found the solution: Fork with corrected behaviour. Works beautifully :slight_smile:

1 Like

Is this a new fork of Extended Open AI conversation?
I’m surprised this thread is dead. This is such a cool tool.
Hey @jekalmin can we make this work with ChatGPT-5-mini?

What does this fork do differently?

Id challenge that since the release of extended oai creation the scripting capabilities of HA + oai default has caught up with it.

Seriously there’s nothing I haven’t been able to do with Friday… Even history stats since 2025.6

What is Friday?