I’ve been working on something super cool lately, and I can’t wait to share it with you all. Drumroll, please…
Introducing my new website that generates automations based on ChatGPT! It’s still very much in early stages, but I’m stoked to get your feedback on it. So, give it a whirl and let me know what you think!
Before you dive in, I recommend checking out the usage instructions and read the Advance section to generate better automations.
Give it a spin and let me know how it works for you. Your comments and thoughts mean the world to me. Together, we can make this automation generator even more awesome and stop writing automations manually!
Thanks in advance for your support, and happy automating!
I didn’t add a progress bar yet so it can take up to a minute max for a response
this is currently the trend that HA is taking so why not add one more step to achieve that end.
to the OP…
how does the user validate that the AI is generating valid code and not (as it has shown to be already doing) just providing “close but not right” code to add to the users confusion of why their automation doesn’t work?
If a user doesn’t know how to write automations in the first place, how will they know if the AI code is correct or how to fix it if not?
Been reading your About page, and here we go again…
The website leverages GPT’s natural language processing to understand your requirements and generate detailed automation scripts.
No it doesn’t. GPT systems don’t understand anything. They build a statistical model of which words normally follow what other words, and then spin a stream of text out of it. Whether the output has any value depends on the prompt and on what they’ve been trained on. Garbage in, garbage out.
Sure, but when their automations won’t work and they’ll ask for support, they should disclose it was generated by an AI as well.
Pretty important to grasp the level of knowledge of the requester about his code…
Say a user asks ChatGPT to write a HA automation. This is a pretty specialist area, so chances are a large proportion of the yaml examples ChatGPT has been trained on come from this forum. This means that:
At least half the examples used contain errors, because they come from the questions forum members are asking, not the answers.
Around 70% of the examples are out of date, because forum pages go back several years.
So. There’s a good chance the AI yaml will be wrong. As @koying says, the users will ask for suport. The incorrect code produced by ChatGPT will be posted on this forum. ChatGPT will use it for training… And so we go on down the rabbit hole.
I love the idea OP! I’ve been using ST/Hubitat/HA for over a decade and this would be a game changer for me. My creativity in useful automations leaves room for desire and even a slightly in need of tweaking automation of my most used and predictable smart sensor available data trends would provoke me to create a smart home that improves the life of the non tech folks around me. The AI haters are getting a little tiresome. You guys are invaluable but you elicit a smell of age old self-conscious jealousy. It’ll be okay if questions arise in this thread. Don’t answer them if you so choose.
It would be great if it worked perfectly, but it’s like self-driving cars — just out of reach, and much more complicated to get right than most people understand.
It’s not even close to that at the moment though, which is why a end up with questions like this, where we had to guess that it was AI:
and this, where we ignored all of the AI attempts and solved the problem in less time than it took to enter the AI prompts in the first place, ending up with code that we understood how it works:
My two concerns is that people will trust the AI “solutions” to be more correct than they might be, missing potentially critical edge cases; and that we will be encouraging a decline in understanding how things really work.
I genuinely fear that this is the tip of an iceberg that will haunt us for years to come. AI should be confined to specific tasks like helping flag up cancer identification in scans.
Nobody is worried about answering questions in your thread. Knock yourself out.
The biggest concern is people won’t limit thier questions to the thread.
One thing I would ask is if it could also spit out a remline saying it was ai generated so when someone inevitably pastes it without reading any disclaimers and it does not work. At least we can see it in thier copy pasta yaml?
Its not about being scared of AI. Heck I sell AI solutions as part of my day job. It’s about knowing what it’s good at. This is something is it not good at… Yet. (I do expect that to change probably within the next 24 months, but we’ll cross that bridge when we get there.)
Right now public LLM models do NOT understand homeassistant. They are date limited to data with dates preceeding current versions of software and were trained using forums that rarely provided correct answers (yay reddit) or answers on depricated products (as of 2018.8.x it was…)
Therefore unlike coding against say, Java where there’s lots of GOOD current references on a reasonably stable platform, HA changes all the time, uses multiple languages internally has an incredibly fast update cadence and many of the public references are flat out wrong. Really, ChatGPT and geminis public model responses for HA automations are pretty awful.
If you could build/train a custom GPT on current versions using a curated list of known good functions and automations relevant to current product and keep it up to date. Use Rag to ground it in reality? Sure. But you’d need funding. Right y, it just isn’t ready for prime time.
It’s still very much in early stages, but I’m stoked to get your feedback on it. So, give it a whirl and let me know what you think!
)
