I am using ChatGPT regularly in many ways in life. In regards to HA, I know exactly what automations I want and how they technically should work, but I recently switched over to HA and therefore my knowledge of HA and yaml is quite weak. I also don’t have the luxury or intention to spent months to learn HA/YAML in depth. Instead for topics like this, I would love to use ChatGPT to deliver code and provide step by step guides on how to do something in HA.
This is what I have been using ChatGPT with other platforms that I want to use, but don’t need to become in expert in quite successfully. But here lies the problem. In comparison to other systems/languages, ChatGPT’s delivers guides and code for HA/YAML with a high rate of errors. ChatGPT refers nearly every single time to old versions of HA where menu items where different or a feature was located somewhere else. And also in regards to YAML, the suggested code constantly violates the current’s version rules. Every single time, when you make ChatGPT aware that this option doesn’t exist or of the YAML error msg, it more or less immediately responds that HA changed the way it works in recent versions and provides the correct steps/code. Unfortunately ChatGPT isn’t that smart that you can tell it to only refer to current versions and as such you waste a significant amount of time going forth and back until you get what you need.
In my mind, part of the problem seems to be frequent changes on HA GUI, but I understand that if something was not user intuitive or evolved, that it required a revamp. But for code, this shouldn’t be the case that newer versions constantly break old codes.
What would need to happen to make ChatGPT and similar tools significantly more powerful in regards to HA? Any insights?
Second what tom_i said: The hard-core or OG HA users still use yaml, it is however no longer a requirement to fully utilise HA…
So use the UI to build your automations, if you still intend to understand the yaml code, you can always view it in the automations you built, themselves.
Training cut-off date for ChatGPT is probably at least a year ago. I’m using gpt-5 and the cut-off date is currently June 2024, so it won’t be aware of any changes since then. (If you don’t know the cut-off date you can ask it.)
High rate of updates HA has changes, new features and fixes every month. Nowadays a lot of these are to do with the UI, but even so there’s a lot ChatGPT won’t be aware of. It may not know much about developments in voice, for example.
Sources HA is a pretty niche field. Most of ChatGPT’s training material will come from this forum, where by definition a large proportion of the posted yaml is wrong - people post it asking for help - and most of the rest is quite old. If you try an LLM which gives you it’s sources, you’ll find nearly all of them are here.
Temperature setting The temperature setting for an LLM controls how “inventive” it can be in its replies. If you’re using ChatGPT through an app you probably can’t change this, but you can ask it to explain what its setting is. A high setting will give you a different answer each time you ask a question.
What you can do about it.
Your prompt Compared with “regular” use of ChatGPT, your prompt when you ask about HA/yaml needs to be long. I mean very long - several pages. Apart from the purpose of your automation/script, you need to explain how you want it written, including details of anything it may not find in its source material that is modern practice - use of the action: key rather than service:, for example. (There are so many examples of the use of service:that it will probably argue about this.) You also need to give a lot of information about your particular system - devices and integrations available, naming conventions and so on. An LLM will not follow links in the prompt - if you want to reference a document you have to paste in the actual text.
Consider integrating your LLM An LLM integrated through Assist will be able to use templates, scripts and aliases to refine its knowledge of your system. You will be able to choose which entities you expose to it and you will be able to use description fields to provide additional information (so you don’t have to put it all in the prompt). There’s some examples here:
Alternatively, you could consider a custom gpt which can read your config and make use of anything it finds there - again this will include comments, descriptions and labels which it can use to expand its “knowledge” of the system, but it will not be able to use scripts or templates as tools and you will not be able to select which devices are exposed to it. There’s an example here. (This particular one should be used with care as it can also write to your config and cause a great deal of damage.)
An integrated approach can produce extraordinary results.
However, it can will involve months of work creating tools and building consistent documentation into your system.
A fascinating project in itself, but much easier to learn yaml.
Thanks for your replies and especially for yours, Jack. The automations I’m building are quite complex and the GUI doesn’t allow for that level of complexity. Looks like I need to bite into the sour apple and learn YAML.
