[Custom Component] extended_openai_conversation: Let's control entities via ChatGPT

And… we can now ask assist to analyze the images / cameras !!!

Here you go, works perfectly ! :smiley:

Require GPT-4 Vision created by @valentinfrlch

This is the spec:

- spec:
    name: vision
    description: Analyze images
    parameters:
      type: object
      properties:
        message:
          type: string
          description: Analyze the images as requested by the user
      required:
      - request
  function:
    type: script
    sequence:
    - service: gpt4vision.image_analyzer
      data:
        max_tokens: 400
        message: "{{request}}"
        image_file: |-
          /media/Allarme_Camera.jpg
          /media/Allarme_Sala1.jpg
          /media/Snapshot_Giardino1_20240425-090813.jpg
        provider: OpenAI
        model: gpt-4-vision-preview
        target_width: 1280
        temperature: 0.5
      response_variable: _function_result

You have to provide the path to your camera’s snapshots (you might want to create an automation to create snapshots every x minutes)

I also adedd to the prompt template:

If I ask you to analyze an image or a camera, use the spec vision

In action:

6 Likes