Context is everything when it comes to getting value from AI

Ask two organisations using the same AI tools why one is compounding gains and the other is struggling, and the answer never comes down to the technology.

What we’re learning as applications of LLMs and agents in the enterprise increase and as spend goes up, organisations are starting to dig deeper into the value they’re getting (or not getting). Most people put the lack of value delivery down to three things:

• “It’s an adoption problem, we just need to roll it out more widely and with a bigger comms push – let’s incentivise token usage”
• “It’s a people problem, they just don’t get it and don’t want to change because it’ll expose their limitations”
• “It’s a use case problem, we’ve not struck gold with the one application that’s going to transform our business yet”

The above are all true, as people underestimate the change management, cultural shift, and human factors and skills needed to use AI tools well. They also overestimate AI’s ability to transform their organisation from nose to tail. I believe these are all symptoms of something more fundamental. And I have a hypothesis:

Context is what determines whether an AI system produces something genuinely useful; something that actually saves human time and effort; something that meets the quality bar your business expects; something that actually delivers valuable outcomes faster, more efficiently, and without waste.

The ability to apply relevant information and discernment is also the thing that most organisations, particularly those carrying significant legacy complexity, haven’t yet got in place. And until they do, the headline productivity numbers they’ve been promised will stay out of reach.

The efficiency figures are real. Maybe,  just not for you. Yet.

When Waracle tested AI augmentation across five distinct contexts over 18 months, the 60% efficiency gain was real — for a single, experienced engineer, working on a clearly scoped task, with no external dependencies, no legacy integration, and no compliance review overhead.

Strip all the complexity out, and AI tools are extraordinary. Add it back in and the architecture that predates current thinking, the processes nobody wrote down, the regulatory obligations that sit around, rather than within the development team… and the number drops. In large enterprise organisations operating in regulated industries, the honest, sustained figure is 10 to 20%.

That’s not a failure. Applied consistently to a team of 50 engineers, 10% is the equivalent of five additional people’s worth of capacity without increasing headcount. But it’s a very different story to tell on a slide, and it requires understanding why the gap exists before you can close it.

This is the context gap.

What legacy organisations are actually dealing with

At a recent executive dinner, a delegate described the challenge their team faces: digitising a business built on decades of process, platform decisions made long before modern cloud architecture, and documentation that exists in fragments across systems, inboxes, and the memories of people who may or may not still be at the organisation.

This is the brownfield reality. And it’s where most enterprise software actually lives.

LLM coding assistants perform well when they can reason across a coherent, well-structured codebase. They perform significantly less well when the codebase is large, inconsistently structured, only partially documented, and full of decisions whose rationale exists nowhere except the institutional memory of whoever built it. Asking an AI to work in that environment without the right context isn’t a technology problem. It’s a setup problem.

Meanwhile, another delegate was discussing their handoff challenge, where  a team creates a proof of concept and then transfers them to business units, with all the complexity and knowledge loss that entails. The people involved immediately identified developing a structured knowledge transfer model as a priority. That instinct is exactly right, and it goes to the heart of what makes AI work at enterprise scale.

Context isn’t just a technical input. It’s the foundation of value delivery.

The first place context has to live is in the infrastructure itself. This means feeding your AI systems with the documentation, architectural decision records, codebase understanding, and domain knowledge they need to produce output that’s actually relevant to your environment, not generic best practice from the open internet.

Organisations doing this well are investing in retrieval-augmented generation pipelines, curated knowledge bases, and agent frameworks where the context travels with the task. One of the more interesting developments in recent times has been multi-agent workflows, where specialised agents for architecture, security, code quality, performance, and operability each bring focused expertise to software development work. The value isn’t just speed. It’s having that expertise applied consistently and comprehensively, at the point where it can actually influence decisions rather than document regrets.

This is a programme of work in its own right – it’s not a hot fix half a day. It’s a reimagining of where knowledge is stored in your organisation, and ensuring that it’s accessible, high quality, queryable and dynamic.

The second challenge is structural. Many organisations are adopting AI tools at the level of individual tasks, a developer using Copilot here, a product owner drafting user stories with an LLM there,  without thinking about how context flows through the system as a whole.

What the evidence from our SDLC research consistently shows is that the biggest gains from AI in enterprise settings don’t come from code generation speed. They come from earlier phases: better requirements definition, stronger technical documentation, fewer late-stage integration surprises. The upstream quality improvement has downstream consequences throughout the delivery cycle.

That value only materialises if the systems are set up to carry context forward. When a product vision is defined, how does that context flow into discovery, definition, design and refinement? When an architectural decision is made, is it captured in a way the AI tooling can access during implementation? 

Without context travelling with the work, the bottlenecks in your value stream persist. Processes aren’t liberated from the  endless meetings with information asymmetry, the “new news” that derails plans, the governance checks that spot errors that require a different set of business context.

The third challenge is the one that’s easiest to underestimate and hardest to fix.

At the executive dinner, a delegate  was running a shadow AI experiment focused on automation, approaching the whole thing from a place of considered scepticism about whether AI was useful at all.

That’s a reasonable starting position, but it’s also a limiting one if it doesn’t evolve into genuine understanding of how to engage with these tools effectively.

The danger in most AI adoption programmes isn’t that people use the tools badly. It’s that they use them at pace without context as a default behaviour, and the output looks fine until it isn’t.

This is a cultural and behavioural shift – a completely new way of thinking and working every day.

The organisations seeing compounding gains are the ones where curious people are doing this work seriously. Training programmes that stop at tool familiarity miss the point. The capability that actually matters is knowing how to bring the right context to the conversation.

What this means in practice?

The honest answer to “when do we see the real efficiency gains?” is: when your context is clean, your systems are designed to carry it, your people know how to work with it, and your organisation is structured to compound the gains.

• Start building the knowledge bases now. 
• Start instrumenting your delivery process now. 
• Start creating the cultural conditions where honest conversation about AI use is normal and expected. 
• Start designing your workflows around context transfer, not just task completion.

But be realistic about the timeline. For organisations carrying significant legacy complexity, the near-term gain is 10 to 20%. That’s meaningful, and it’s real. The path to ‘more’ runs through the context problem… and there’s no shortcut through that work.