Fine-tuning an LLM to write docs like it's 1995
A hands-on experiment fine-tunes small local LLMs to imitate 1990s Microsoft technical documentation style.
The author builds a corpus from old Microsoft manuals, cleans OCR text, generates instruction-style JSONL examples, and fine-tunes Llama 3.1 8B and Qwen 2.5 7B with QLoRA. Tests cover malloc(), a fictional Win32 API, and a deliberately anachronistic REST API prompt. Qwen fine-tunes transfer the period documentation style best, but the experiment also shows hallucination risks, tuning complexity, and why these models augment rather than replace technical writers.
This article documents the author's experiment in fine-tuning a small local LLM to write "like a software technical documentation author from the 1980s and 1990s." The author first obtained OCR text from Microsoft's old documentation collections on Bitsavers and the Internet Archive, with sources spanning obsolete manuals and SDK documents from 1977 to 2005, totaling over 37 million words. The data was first cleaned with Python to remove indexes, prefaces, and OCR noise, then a cheap gemma-4-26b model on OpenRouter was used to classify whether each paragraph was usable. Finally, the text was split into roughly 512-token chunks by paragraph, section, and code block, and paired with templates to generate synthetic instructions, forming 192,456 JSONL training samples. The author emphasizes this is personal, non-commercial research, and that the corpus and adapters will not be distributed.
Free shows the 3-line summary; Pro unlocks the full deep summary (~300 words) so you never have to click through.
See Pro plans →Want the original English / full article?
Read on Hacker News (AI keywords) →Related
Summaries are AI-generated; the original article is authoritative.