Data is the bottleneck in the agentic era

A capable agent can still act on yesterday’s data. Once the model can do the job, I think that becomes the next bottleneck: delivering dependable inputs, and keeping up with the records the agent’s actions create.

What I mean by an agent

The word “agent” has become vague, so here’s the frame I use. Think of an agent as a capable coworker inside a governed company. A coworker has an identity, skills, tools, knowledge of how the business works, and someone they answer to.

An enterprise agent pursues a goal someone delegated to it, and takes actions that change business systems. Like that coworker, it needs defined authority, oversight, and someone accountable for its work.

The frame matters because it changes what you optimise for. A coworker needs clear data, reliable tools and defined authority. Rules approve what’s within policy, and an authorised person handles exceptions. So does an agent. With that in place, here’s the problem. I’m looking at the tables and pipelines agents read, and the records their actions produce. Business systems still own the facts.

The data problem has two sides

Going in, good data makes good agents. That has always been true for machine learning. What’s changed is the cost of a mistake.

In the dashboard era, a bad row in the warehouse had a person in front of it. An analyst might spot a $10,000 anomaly in a report and correct it, or ignore it. Either way, the reasoning stayed in their head. The data recorded what happened, but rarely why.

Without checks before it acts, an agent can turn a bad row into a wrong refund or a misrouted shipment, at machine speed. So the data platform has to expose how fresh data is and what it means. And the workflow has to check authority, and pause when it can’t tell whether an action is safe.

If you build agents, you’ve probably felt this. The agent works in the demo, on clean sample data. In production it hits stale records, ambiguous schemas, and tables that look right but aren’t. You blame the prompt. You tune the retrieval. But the cause is upstream: the data was never built for a consumer that acts on its own.

Coming out, agents create records the platform has to handle. Tool calls, checks and retries can each leave a trace. For a fleet of agents acting all day, I’d check whether the platform can capture those records and make them available within the workflow’s deadlines.

Volume isn’t the whole story. This is decision data, not server logs or clickstreams. How quickly it must be available depends on the workflow. Retention and access rules already exist; the work is applying them to agent decisions too. A useful record connects the inputs, the checks and the action, so someone can investigate what happened. And it feeds back into the system that produced it.

That creates a loop. Agents use data to decide. Their decisions create more data. That data shapes the next decisions. If the data layer can’t keep up, because ingestion is slow, transformations are brittle or data arrives late, the whole loop stalls.

The agentic feedback loop

Read agents use data
Decide and act
Record decisions become data
Feed back into the next decision

if the data layer can't keep up, the loop stalls

Once a model can do the job, I think reliable data delivery becomes the next constraint.

Why this is easy to miss

Most of the agent conversation is about the intelligence layer: which model, which prompt, how to chain the reasoning. That’s rational. Model quality matters, and it deserves attention. Building reliable orchestration, evaluation and tool integration is real, hard work.

But models are improving quickly and getting easier to access. For many enterprise tasks, in my view, they’re already good enough. Once that’s true, the bottleneck moves. Teams that invested only in the intelligence layer hit a wall: the models are capable, and the data isn’t ready.

The two layers depend on each other. Agents rely on the data layer for context, freshness and correctness. And once agents act, the platform also has to capture the records they produce. Each one limits the other.

When an agent builder spends days tuning a prompt to work around stale or ambiguous data, that’s the dependency breaking. When a data engineer builds a pipeline without realising its consumers now act on their own, that’s the same break from the other side.

The data layer doesn’t improve by itself. It takes deliberate engineering: streaming ingestion, schema contracts, service levels, cost controls, freshness targets. None of it demos well, but it matters the moment an agent depends on a table being fresh enough to act on.

I spent years building data platforms: lakehouse architectures, Spark pipelines, distributed compute. The challenge there was determinism. Move data reliably from A to B, transform it correctly, deliver it on time. Failures included schema mismatches, resource limits and skewed partitions. The agentic world keeps all of those problems and adds new ones. The platform now serves decision-makers that run around the clock and produce their own data, which has to be captured and processed as it happens.

An example

Take a made-up company, Shiba Resorts, a Hawaiian resort that sells vacation packages, hotel stays, spa bookings and guided excursions. Its platform captures reservations, guest preferences, seasonal prices, cancellations, payments and review sentiment.

Today that data feeds dashboards and a few periodic models. The data team builds the pipelines, keeps them fresh and watches the cost. It works.

Now add agents:

A pricing agent that adjusts package rates as demand changes.
A fraud agent that flags suspicious bookings before payment clears.
A personalisation agent that tailors packages to each guest, from past stays, dietary needs and activities.
A concierge agent that answers questions and suggests excursions based on weather and availability.

Each one reads from several pipelines, and each one produces decisions of its own. Suppose the pricing agent reads yesterday’s availability and proposes a discount for a nearly full hotel. The workflow should check freshness and pricing limits before the rate changes, and record afterwards whether the change went through.

So the data didn’t just grow. It changed character. It’s faster, the stakes are higher, and it flows both ways: agents read from the data layer and write back to it. The team that ran batch pipelines with hourly deadlines now runs the platform the agents depend on. If it can’t keep up, the agents stall, and the business loses the advantage it was after.

The shift underneath

The intelligence layer has real scaling problems: cost, latency, evaluation. But even when a better model arrives, each team still has to maintain its own schemas, permissions and freshness targets. The data layer scales differently. It takes engineering, investment and design decisions that build on each other over years. There’s no shortcut.

That asymmetry is an advantage for whoever sees it early. Not because they’ll have better models, since most teams will have access to similar ones, but because their data platform can feed agents reliably and absorb what those agents produce.

This isn’t an argument that data matters more than models. They rise and fall together. Right now, the data layer is the one getting too little investment.

Before tuning the resort’s pricing agent again, I’d check whether it can see current bookings, and whether its last rate change was recorded. That’s where I’d start looking for the next bottleneck.

This is part 1 of The Agentic Data Stack. In part 2, I describe what an AI-ready data platform looks like, and why the lakehouse is a natural foundation for it.