Why can't we have LLMs writing documentation?

Question

Everyone says documenting code is important yet no one seems to want to take the time to write such documentation. My question is wouldn't it be possible to just let a LLM keep reading your repo and generate the necessary documentation? People already use LLMs to code and are trying to make LLMs work as full developers. If we expect them to work as independent developers in the near future, can't we get them to at least write useful documentation first?

jfengel · Accepted Answer

I'd break things down into two broad kind of documentation:
1. Comments on code
2. User-facing manuals
There's no need for an LLM to write the former. Comments are left by a person to explain decisions when they're no longer present. LLMs don't go away. You can always ask the LLM why the code was written that way.
As for users... they famously don't read. UX designers take that into account, and try to make their interfaces self-documenting. A user who can't figure it out from the design in front of them probably won't find the right part of the manual to read, either. They'd rather dive in than read cover-to-cover.
The "for dummies" books seem to sell well. I wonder if you could get an LLM to write those.

maxbaines · Answer

This is the first thing I do when using LLM's to create code. So we can and I do have LLMs writing documentation.In my domain IIoT, I even created a yaml structure to make this easy. https://unsframework.com (wip)

beardyw · Answer

Writing documentation after coding should strongly feature why things were done the way they were. How could an LLM do that?

dave4420 · Answer

I expect the LLM would need more context to make the documentation useful for readers.

austin-cheney · Answer

Good documentation is no different than good software architecture. Neither are challenging yet most people in the work force cannot do either effectively, because these things require extensive practice making original organizational decisions. Practice isn&rsquo;t something that can be wished away by some ineffective hallucination tool.If you think that lack of experience can be replaced by AI then good luck. Microsoft has very recently lost all faith in AI to do any of this for reason.