Engineering notes
Building documentation that has to prove itself: what works, what breaks, and what it costs.
Free tokens change the design: porting a cost-optimized LLM pipeline to local models
Our cloud pipeline is tuned to minimize token cost. On a local GPU that optimization is backwards. Here is how inverting it took an 8B-class model from thin, uncited pages to a median of twelve verified citations per page.
Read postOllama silently truncates your prompts, and your only symptom is bad output
Ollama defaults to a 4096-token context window and cuts longer prompts without an error. How it degraded our pipeline, how to detect it, and how to fix it on every platform.
Read postWhy we open-sourced RepoFold: some code is never allowed to leave the building
The most common objection to RepoFold was never about quality. It was policy. So we released the pipeline as open source, running fully offline on local models, with citations that are correct by construction.
Read postHow I force an LLM to cite every sentence it writes about your code
Prompting a model to "cite your sources" does not work. Here is the validation pipeline that makes generated documentation verifiable, and what it costs to keep it honest.
Read post