Article • Artificial Intelligence • Development
11 November 2026

Evading the Forensics Tax – Using AI to map business logic across sprawling estates

Our Head of Engineering, Lachlann Rattray, explores how building Open Knowledge Format LLM Wikis could increase system understanding, aid in estate planning and reduce the ‘forensics tax’ in complex multi-repository systems.

Evading the Forensics Tax Article Hero Image

At one of our recent roundtable events, I led a discussion exploring how organisations can leverage large artificial intelligence budgets not just to write code, but to map the labyrinth of legacy estates. I had a particular use case in mind – a wide digital estate consisting of numerous repositories, patchy documentation and concentrated SME knowledge. 

As I was casting around at Waracle for ideas, our Director of Engineering, Data and AI Ewan Nicolson sent me a Gist written by Andrej Karpathy. This article is about what I learned from trying to apply this ‘LLM Wiki’ approach to a codebase in the hope that by deploying agent-driven workflows and an LLM wiki architecture, engineering teams can systematically untangle tribal knowledge, reduce the ‘forensics tax’ of modernisation, and mitigate regulatory risk.

Unpacking the ‘forensics tax’ in multi-repo estates

I mentioned above that the context for the problem I’m trying to solve lies in trying to work effectively within a large, sprawling estate encompassing hundreds of repositories. Institutional knowledge in these environments is often concentrated among a handful of subject matter experts, creating a severe operational bottleneck. In my specific case, this led to a particularly inefficient chain of events when trying to deliver new work:

  • New feature requests come into the engineering teams. These are often fairly well understood in terms of user outcomes, however how they fit into the current ecosystem is less well understood.
  • This lack of understanding  triggers a heavy ‘forensics tax’, requiring extensive manual investigation just to determine the current state of the application before new code can be drafted.
  • This causes significant wait time at the start of the delivery funnel, as technical leads and business analysts wade through source code and stored procedures to try to understand how this new work fits into the existing systems. 

Now, if we are dealing with a system with just a single mono-repo its likely that using Claude Code to query the code base as we build is sufficient. However for larger estates were user journeys and business logic is affected by the implementation details spread across multiple repositories this approach can break down, context windows fill quickly and the quality of answers we get back is reduced.

Multi-repo estates

The LLM wiki architecture and Open Knowledge Format

Karpathy’s original Gist suggests an alternative approach to RAG when trying to query or get answers from a base set of source material. Although RAG can be successful, it requires re-synthesising knowledge from snippets found at query time. When attempting to navigate a vast, multi-repository system, we quickly saturate the model’s context window and witness a sharp rise in hallucinations. 

The wiki approach solves this by compiling the knowledge beforehand. The AI reads the raw repositories once and synthesises the business logic into a persistent, interlinked network of markdown pages. It starts with root-level entity pages and then builds a layer of concept, comparison and overview-style pages on top. Everything is in Markdown and we use MD links as references between pages to make the system navigable. 

This approach means that you aren’t forcing a model to read raw code repeatedly; instead, you are querying a structured, compounding map of your system. This method should be cheaper, entirely searchable, and, perhaps most importantly,  leaves your team with a readable artefact they can actually debate and verify. 

About three months after Karpathy posted his Gist, Google took the idea and created a loose standard which it calls ‘Open Knowledge Format’.

LLM Wiki and OKF

Taking the original Gist and some of the extra thinking and examples that Google put behind the OKF format, I started planning how I might apply this to the problem space described above. I started by running a short POC to see what kind of wiki artefact we could produce on a simple repository.

I decided to test the  approach on an open-source React and Postgres shop starter kit, using a custom Claude Markdown file to establish guidelines for compact, high-signal pages and clear sources of truth for every claim. 

The foundational structure included:

  • An index file serving as both a table of contents and the primary entry point for natural language queries.
  • A log Markdown file to track changes, navigable Markdown links between concepts, and open-question sections to track the AI’s confidence levels.
  • Custom automated skills for repository ingestion, including an ingest table skill, an ingest service skill, a health-check linting skill, and a query skill.
Evading the Forensics Tax Article Image 1

Before I started to build the wiki out, I came up with five questions that I would ask of both the raw source code (via Claude) and of the wiki only. 

Once I had compiled the wiki (which took about an hour or so), I decided it might also be interesting to ask the same questions to the Claude Code context with which I had built the wiki. 

In total, the wiki cost about $13 to build, using a combination of Sonnet 5.5 and Opus 4.8. Running my five questions against the wiki only cost $0.30, and against the raw source code was $0.51. This is not a significant difference, but that is to be expected when the source code was relatively small. I would expect the difference here to be more significant as the size of the codebase increases.

Asking my five questions of the LLM Wiki, the source code, and of the Claude context which had built my wiki produced an interesting result. The first four questions produced answers which were pretty consistent with each other. The final question, which was along the lines of ‘can a user favourite the same thing twice’, produced some divergence. The builder context answered correctly: no, this is not possible due to the behaviour of a 3rd party NPM package. The raw code answer was incorrect, answering that ‘yes’, this was possible. The wiki only answer came back with a useful ‘I’m not sure’, but consequently it was able to identify the boundary of its knowledge and provide direction for further research, basically by saying I would need to go and investigate how a particular NPM package worked.

Evading the Forensics Tax Article Image 2

Next steps: Estate level documentation

While documenting a single repository is valuable, it’s at the estate level that I think this approach might have the biggest impact. The next steps for this POC are to apply it to a modest multi-repository estate. To do this, we can adopt a similar approach to the single repo proof I described above, however, we will need to introduce an additional federating layer of articles at the estate level, which will document estate-level concerns such as cross-cutting concepts like authentication, as well as inter-repository contracts and dependencies. 

The real opportunity here goes beyond just understanding business logic in discrete areas of the estate, but rather in understanding the spaces between code. Business logic in estates like this becomes fragmented across API contracts, shared libraries and event buses. The LLM Wiki doesn’t just read a function in Repo A, it also connects it to downstream consequences in Repo B or in the IAC code base. This allows for more detailed impact analysis when planning changes to the system.

In order for this to be useful to delivery teams, it will need to be stored centrally and accessible behind an MCP server, which would allow users and agents to make estate level queries from within their local workspaces. 

Another consideration will be making sure that the wiki does not become frozen in time. As the system evolves, so too must the wiki. We can manage this through our CI/CD pipelines by either updating the wiki as pull requests complete or on a scheduled basis. 

Finally, introducing a mechanism for human review, amendment, and confidence will be important. Experimenting with having the LLM wiki generation process include a ‘confidence’ level will be interesting. We can also adopt the OKR ‘verified’ tag to allow us to capture when a human has reviewed a particular article. It then may be useful to include an ‘accuracy’ score when giving users answers, indicating that the answer was based on a certain number of verified articles. 

To summarise…

Start with one repository, not the estate. Prove the shape, the cost, and the quality on something contained before federating.

Get ready to find bugs and inconsistencies. A well-designed set of wiki-building skills and prompts will help to surface inconsistencies as well as bugs from your source code. 

Keep the map alive. A wiki written once decays like any other documentation. Feed it from Git logs, user stories and existing documentation, and refresh it on a trigger, whether nightly, weekly or as part of the CI/CD pipeline. Cache frequently asked questions back into concept articles so the wiki improves with use.

Human ownership. The engineering team need to own this, not just be ‘looped in’. One of the interesting aspects of this approach is that in the end we have a human readable set of mark down files. Using OKF verification tags and LLM derived quality scores we can be transparent about the answers the system gives. The wiki becomes a living tool collaborated on by Humans and Agents.

Make confidence visible. Open-question sections turn “the AI wasn’t sure” from a hidden failure into a work queue for your experts.

What I’ve shown here is one small experiment, not a method proven at scale, and the hard parts are organisational: ownership, escalation and trust.

So here is the question I would put to engineering leaders with a wide, complex digital estate and an AI budget. If your most experienced engineer left tomorrow, could the estate still explain itself?

If you are weighing up a modernisation programme and would like to talk this through, get in touch today.

Share this article

Authors

Lachlann Rattray
Lachlann RattrayHead of Engineering

Related