Fun Fact for the Day! - A Cautionary Tale for the use of Gen AI

chairstacker · February 13, 2025, 9:50pm

I thought I’d have a ‘Fun Fact for the Day!’ automation that gives Google’s Gen AI this instruction once a day and then directs it to be transmitted through the speaker in the living room:

“Pick an A-List actor whose birthday it is today and respond with a sentence in the following structure:
Fun Fact for the Day - Did you know it’s <Actor’s Name> birthday today? <He/She/They> are turning XX years old and one of their best known roles was in <the name of movie or TV show”>

This is the info I receive today:
“Fun Fact for the Day - Did you know it’s Elizabeth Olsen’s birthday today? She is turning 36 years old and one of her best known roles was in WandaVision.”

And while Elizabeth Olsen actually was in WandaVision, her 36th birthday is not until Feb 16th

And this is what I received yesterday:
“Fun Fact for the Day - Did you know it’s Keanu Reeves’ birthday today? He is turning 58 years old and one of his best known roles was in The Matrix.”

He was born September 2, 1964 - so: nope, it wasn’t his birthday yesterday either and his 58th was more than 2 years ago.

Sir_Goodenough · February 13, 2025, 10:18pm

And you are surprised?

chairstacker · February 13, 2025, 11:40pm

Nope, just annoyed - it could be so easy

d921 · February 13, 2025, 11:42pm

The day it’s “easy” for AI to replace your human mind…

chairstacker · February 28, 2025, 7:14pm

Just for the fun of it - here’s a quick update of the ‘Fun Fact of the Day’ that Google’s GenAI had to share:

February 27, 2025
Fun Fact for the Day - Did you know it’s Halle Berry’s birthday today? She is turning 57 years old and one of her best known roles was in X-Men.

February 26, 2025
Fun Fact for the Day - Did you know it’s Kate Winslet’s birthday today? She is turning 48 years old and one of her best known roles was in Titanic.

February 25, 2025
Fun Fact for the Day - Did you know it’s Halle Berry’s birthday today? She is turning 57 years old and one of her best known roles was in X-Men.

February 24, 2025
Fun Fact for the Day - Did you know it’s Ryan Reynolds’ birthday today? He is turning 47 years old and one of his best known roles was in Deadpool.

February 23, 2025
Fun Fact for the Day - Did you know it’s Kate Winslet’s birthday today? She is turning 48 years old and one of her best known roles was in Titanic.

February 22, 2025
Fun Fact for the Day - Did you know it’s Ryan Gosling’s birthday today? He is turning 42 years old and one of his best known roles was in La La Land.

February 21, 2025
Fun Fact for the Day - Did you know it’s Kate Winslet’s birthday today? She is turning 48 years old and one of her best known roles was in Titanic.

February 19, 2025
Fun Fact for the Day - Did you know it’s Kate Winslet’s birthday today? She is turning 48 years old and one of her best known roles was in Titanic.

February 18, 2025
Fun Fact for the Day - Did you know it’s Kate Winslet’s birthday today? She is turning 48 years old and one of her best known roles was in Titanic.
February 17, 2025
Fun Fact for the Day - Did you know it’s Meryl Streep’s birthday today? She is turning 74 years old and one of her best known roles was in Sophie’s Choice.

February 16, 2025
Fun Fact for the Day - Did you know it’s Robert Downey Jr’s birthday today? He is turning 58 years old and one of his best known roles was in Iron Man.

haus · February 28, 2025, 10:01pm

Kate Winslet sure ages fast. Can you ask ChatGPT the same thing each day and see which one is more correct?

chairstacker · February 28, 2025, 10:07pm

These are the results of the once-a-day run on the shown dates.
It’s scheduled to run during dinner time and I’m not turning it off because it amuses us - we might be easily amused, though

And yes, it’s “Kate Winslet’s birthday” every other day, but it’s always the same age - I hope, she sincerely appreciates this.

NathanCu · March 14, 2025, 12:06am

Chair. I’d be interested in the debug output of the conversation here to see exactly what part of the chain saw what…

Possible? Or enough of one to get the context?

Heh. Context.

chairstacker · March 14, 2025, 1:01am

Hi @NathanCu, I tried to get the debug info from the integration y enabling debug logging here

but it doesn’t show anything in the downloaded logs.

Same, when I add these lines into the configuration.yaml

logger:
  logs:
    homeassistant.components.conversation: debug
    homeassistant.components.google_generative_ai_conversation: debug

I then launched the Google Generative AI popup and submitted the same question as I did from the script, i.e.

Doing that a few times got me these results:

Fun Fact for the Day - Did you know it's Billy Crystal's birthday today? He is turning 77 years old and one of his best known roles was in When Harry Met Sally.

Fun Fact for the Day - Did you know it's Jamie Bell's birthday today? He is turning 39 years old and one of his best known roles was in Billy Elliot.

Fun Fact for the Day - Did you know it's Common's birthday today? He is turning 52 years old and one of his best known roles was in Selma.

Fun Fact for the Day - Did you know it's Danny Masterson's birthday today? He is turning 49 years old and one of his best known roles was in That '70s Show.

Fun Fact for the Day - Did you know it's Kaya Scodelario's birthday today? She is turning 32 years old and one of her best known roles was in Maze Runner.

Also some of those folks’ birthdays are actually March 14th, the results are acceptable, me thinks.

Being encouraged by these responses I went back and ran the script again a few times - with the same disappointing, incorrect results as before - and: yes, Brad Pitt was returend twice in a row

Fun Fact for the Day - Did you know it's Robert Downey Junior's birthday today? He is turning fifty-nine years old and one of his best known roles was in Iron Man.

Fun Fact for the Day - Did you know it's Jamie Foxx birthday today? He is turning 56 years old and one of his best known roles was in Ray.

Fun Fact for the Day - Did you know it's Brad Pitt's birthday today? He is turning 60 years old and one of his best known roles was in Fight Club.

Happy to run that a few more times and provide more debug info - you’d just need to guide me to where I can get these entries.

NathanCu · March 14, 2025, 1:26am

Voice Assistants:

Pick your pipeline:

Debug:

…Will bring you here - have fun storming the ca… exploring the pipeline…

chairstacker · March 14, 2025, 1:43am

Sorry, misunderstanding, I guess

It’s not a Voice Assistant issue I’m facing here, it’s an issue with the Google Generative AI integration at http://homeassistant:8123/config/integrations/integration/google_generative_ai_conversation

So, there’s no voice pipeline.

I run another script against this integration which checks if my garbage cans have been moved by the trash company based on two images I’m sending - that seems to be working much better that the Fun Fact one above:

alias: Google Gen AI Test
sequence:
  - action: google_generative_ai_conversation.generate_content
    data:
      prompt: >-
        Very briefly describe the major differences between these two images,
        focussing on the two trash cans.  Does it look like either of the trash
        cans has moved or has been turned?  If so, how? Your message needs to be
        short enough to fit in a phone notification. If there are no trash cans
        in either of the images, point it out specifically. Ignore and do not
        start the notification with "Here's a short message summarizing the
        image differences:" Consider the change of the sun shining on the
        garbage cans in your analysis, but do not comment on it.
      filenames:
        - >-
          /media/SecurityCams/1_OutdoorCam/{{
          states('input_text.trashcan_reference_timestamp') }}.jpg
        - >-
          /media/SecurityCams/1_OutdoorCam/{{
          states('input_text.outdoorcam_snapshot_time') }}.jpg
    response_variable: generated_content

NathanCu · March 14, 2025, 1:48am

Ahhh ok yeah. I have no clue how to help with that. I’ve been trying to avoid those mostly except for utilitarian uses like summarized cameras… (same api socket same call just heavily restricted toks)

What I can offer is. Start small with a very simple ask. Give clear robust instructions.

If you do that and it still bombs
and youre sure the model at least supports what you’re trying to do. I’d have already moved on to a different model?

chairstacker · March 14, 2025, 2:14am

No problem - that was my approach and it seems to work fine for image analysis (I have a few more of these automations/scripts set up to analyze images from my driveway and my yard) and it seems to work mostly fine for that.

Just when I ask for some factual information - rather than interpreting image deltas/movements - it seems to ‘not perform that well’.

And: yup, I agree - I might have to move to another model for my ‘Fun Fact for the Day!’

chairstacker · March 15, 2025, 12:28am

Update:

Seems like, that without a reference, Google AI doesn’t know what day it is today:

action: google_generative_ai_conversation.generate_content
data:
  prompt: >-
          What day is today?
          What day is it tomorrow?

Response:

text: >
  I do not have access to real-time information, including the current date. To
  find out what day it is today and tomorrow, please check a calendar or search
  online.

Adding in a reference like this:

action: google_generative_ai_conversation.generate_content
data:
  prompt: >-
          {{ now() }}
          What day is today?
          What day is it tomorrow?

produces this response

text: |
  Today is Friday, March 14th, 2025.

  Tomorrow, March 15th, 2025, will be Saturday.

I’ll try this in the daily fun fact automation:

data:
  prompt: >-
    Which A-List actor has a birthday {{ now() }}? How old is the actor {{ now()
    }}?  What the actor's most famous movie role? Provide the response in the
    following format

    Fun Fact for the Day - Did you know it's <Actor's Name> birthday today?
    <He/She/They are turning XX years old and one of their best
    known roles was in <the name of movie or TV show> directed by <director's
    name>

photo64 · March 16, 2025, 1:25am

Interestingly I tried your prompt by typing it in the assist prompt and when I included the words “A-List” it consistently gave me incorrect results however when I took that out it started giving me correct results. Just curious, are you just calling this through an automation using a sentence trigger such as “Fun fact of the Day”

chairstacker · March 16, 2025, 1:35am

If I type it into the Google Gen AI Assist prompt as shown above, I get a proper result.

If I send it through an automation or script, I need to define today’s date, or the system picks a random one for me - actually: thinking about it, the date doesn’t seem very random, just wrong.