For those not aware, OpenAI has a feature that allows you to create “Custom GPTs” which are basically instances of GPT-4 that you can provide documentation it can reference when you ask it questions, and every time you start a new conversation it can still access those documents.
One use case for this is to literally just give it a text file, pdf, etc. with a bunch of software documentation it can search through. I suspect this will be an extremely common use case for AI models in the future as well.
My idea is this: Take all the Home Assistant documentation and create a huge “unified” document with all of it appended together. Considering that the documentation is on GitHub anyway as markdown, this could theoretically be done programmatically and automatically with every single change. Either hosted on GitHub itself, or hosted externally that pulls from the repo (as to not have an entire duplicate copy of the info within the same repo). In either case it could probably be accomplished with a GitHub action.
Some additional miscellaneous notes:
Right now the Custom GPT feature is only available to chat GPT Plus users. Anyone who creates a custom GPT can share it publicly with a link that others can use, but again only those with Chat GPT Plus can use those shared ones.
Anyone who has chatGPT Plus can create their own custom GPTs really easily by just uploading the document, so it’s not like Home Assistant would have to host an official one.
If there was to be some kind of official documentation support bot, I think there are other products by OpenAI that would be better suited, but they would cost money. So for my idea, at the moment, I’m just imagining it would be used by individual users creating their own custom GPTs.
ChatGPT currently answers questions about Home Assistant. Obviously it’s getting this information from somewhere. If it’s getting it from anything that can be scraped from the internet then it has access to the official documentation.
If that’s the case then, based on the poor quality of some of its responses that users have posted (especially the error-ridden scripts and automations it has created), it’s not a reliable reference.
It seems to creatively ‘free style’ with available information and produces authorative-sounding but flawed examples and explanations.
Perhaps if its available resources are limited to just the official documentation, as you have suggested, then it would constrain its “creativity”. However, I have my doubts; it seems able to invent things, right and wrong.
Yes that’s basically my point. Since home assistant documentation probably didn’t show up very often in the training data (possibly even only once from the documentation website itself), it’s not going to be reliable at all. Not to mention the data is going to be more out of date with every release.
So by giving it the latest documentation to reference explicitly, the reliability should be infinitely higher.
Also I should point out the distinction between GPT-4 (the paid one) and GPT 3.5 (the free one). GPT 3.5 is indeed unreliable and I never use it even when I hit the rate limit on GPT-4. I consider 3.5 completely braindead in comparison to GPT-4. So not sure if you’ve only ever used the free ChatGPT, but if so just know it’s a totally different animal than GPT-4.
There are only a few examples in the official documentation and some of them use syntax that presupposes an understanding of where the YAML should be added (such as in the configuration file or in a separate file dedicated to the example’s integration). Given the paucity of examples (compared to, say, the number of examples in this forum), I suspect even GPT-4 might be less than infinitely higher in its reliability as an authoritative tutor.
When it comes to the accuracy of ChatGPT’s Home Assistant suggestions, it has yet to rise to the level of good.
To be clear, I’m not a gatekeeper for Feature Requests and just expressing my doubts about the claimed benefits this would offer.
If money is involved in operating this proposed feature (paid account for GPT-4) Nabu Casa will need to be involved so the decision is their’s to make. In other words, volunteers alone can’t sustain this FR (unless they wish to fund it themselves).
