When Data is for Agents, Not Humans

The Back Gate

The workshop on trusting AI agents began with an AI agent getting its human lost. Anand asked Google Maps how to reach Microsoft's Luxor North Tower in Mahadevapura. Maps confidently routed him to the back gate. The security guard at the back gate was unmoved: the front gate, please — twenty-two minutes away. And so the man billed as an LLM Psychologist walked into his own session ten minutes late, apologising to a room of thirty data practitioners who had paid to learn how much they should trust machine advice.

He didn't hide the lesson in it. Hours later, mid-discussion, he came back to that walk unprompted: a restaurant recommendation that turns out too sweet costs nothing, so don't bother verifying it. A navigation error that makes you late to your own workshop — that has real-world costs, and deserved a check. Trust isn't a setting; it's a budget, priced by the cost of being wrong.

"I missed my morning walk; I didn't miss my afternoon walk."

— Anand, apologising for his late arrival

Anand — co-founder of Gramener, now at Straive — was at The Fifth Elephant, Hasgeek's data community, to run a four-hour hands-on workshop built on a deceptively simple observation, one he'd trailed on his blog a week earlier: for thirty years we cleaned data for humans. Now agents are reading it.

"The premise here is that earlier humans were in some shape or form the consumers of data. Software became a big consumer of data. Now agents are becoming a big consumer of data."

— Anand, opening the workshop

We know how to organise data for humans (dashboards, folders, documentation portals) and for software (schemas, APIs). Nobody yet knows how to organise it for agents — and Anand's wager was that anyone claiming to know is already out of date, because the agents change monthly. So this would not be a workshop that hands out answers. "This is a 'teaching you to fish' rather than 'giving you a fish' kind of a workshop," he said. The plan: figure out how to figure out whether any given way of organising data for agents actually works.

And in a twist that set the tone for the afternoon, the participants wouldn't be doing the figuring either. "This is a workshop; you're going to be doing most of the work. Actually, you are not going to be doing most of the work; you're going to be delegating most of the work to AI. That is what we are here to learn. And what is left, what it cannot do, is broadly what we will be doing."

Twenty-Two Hunches

Before any AI was consulted, Anand made everyone commit to a guess. A form went up — seven questions, filled in progressively through the afternoon — and the first question was bait for the ego: how should we organise data differently for agents versus humans? What do you think?

"The point is not to use AI to get to the answer. The point is not to get to an answer; it is to set a prior."

— Anand

Twenty-two hunches came in (read them all). "Data should be organized as graphs." "MARKDOWN ALL THE WAY!!" "JSON or XML should perform better than tables or CSVs." "In form of trees :) so that relationshiop ships can be associated" — a typo the room would have enjoyed. One participant offered the most quotable prior of the day: "Separation of concerns is not a natural behaviour for an AI agent." Scanning the word cloud, Anand noted that "context" and "metadata" kept surfacing — the room's collective gut said explain the data better.

To each hunch he offered the same gentle counter: fine, but wouldn't we do that for humans too? Semantic meaning, catalogs, markdown — all plausibly good for everyone, which makes them uninteresting as answers to this question. The interesting claims are where agents and humans genuinely diverge. "For every one of these points, because they are valid, I'm also offering a counterpoint. My main idea is, let's test. And then we will know for sure. Until we test, we don't know."

A participant pushed back early: if we just ask ChatGPT what's good for LLMs, aren't we just reading the internet's average opinion back to ourselves? Anand's answer was disarmingly honest — where you're the expert, overrule the machine; everywhere else, you have no basis to judge it, and "on average, there are more things that the agent knows about than we know about." He was explicit that this was a chosen stance, not a law:

"I'm taking the non-expert perspective here for two reasons: I believe that over time agents will get smarter — they certainly seem to be continuing to get smarter — and the number of areas in which we will be able to have deep expertise is small."

— Anand

Someone else worried the question was too broad — data covers PDFs, tables, logs, everything. That drew out one of the day's most practical heuristics for staying current in a field that resets monthly:

"I start with something tougher than I believe agents can solve, see if it really fails and totally messes up, and then I know: okay, fine, this really is not possible. Every now and then — happens more than half the time — I find that something that wouldn't have been possible last month is now suddenly possible."

— Anand, on posing harder and harder problems

A Fleet of Agents Does the Reading

Then the delegation began. A QR code led to a research prompt — itself written by an AI, steered by Anand in a meta-prompting session (original chat ↗) the previous week. It asked each agent to search the web for current best practices in organising data for agents (structured data, documents, code, logs — but explicitly excluding harness artifacts like AGENTS.md and SKILL.md, which Anand considers processing machinery, not data), then return the top ten techniques as standardised cards, each with a mechanism, a testable claim, and evidence marked measured or opinion.

The room's arsenal was a census of the 2026 agent landscape: Claude Opus 4.8 on High, GPT-5.5 with deep thinking, Gemini 3.1 Pro, Codex, Copilot with Opus, MiniMax M3 on xhigh in a Hermes agent, DeepSeek-V3.2, Pi, even KiloCode — a tool Anand cheerfully admitted he was hearing of for the first time. (Gemini users got a gentle warning: for this kind of work, "Gemini doesn't do as good a job today" as Claude or ChatGPT.)

Seventeen research reports came back and were pasted into the form. Anand fed the pile through a collation prompt with a nice methodological touch: score each technique on evidence, mechanism and testability — but never on how many submissions mention it, because "popularity may just mean good SEO, not good evidence."

The collation run (original chat ↗) counted fifteen distinct submissions in the pile and distilled them into ten ranked technique cards plus 22 runners-up (see the full list). Behind the winner came executing queries outside the context window (return only the small final result, not the intermediate tables), schema-constrained tool interfaces, search-and-filter before listing (in 13 of 15 submissions), and concise, bounded tool outputs — with crowd favourites like markdown-everywhere and llms.txt relegated to the runners-up bench for weak evidence despite high popularity.

The winner, appearing in 11 of 15 submissions, was progressive, just-in-time access:

"For humans, assemble all plausibly relevant documents, schemas, etc. in one portal before the work begins. Whereas for agents, provide a compact catalog of paths, URLs, metadata, query handles, and let the agent search and fetch selected content only when needed."

— The collated technique card for T1, read aloud by Anand

"There's no way I would have guessed," Anand admitted — he "would not have put this on my list." The room, though, wasn't universally impressed. Amit argued it wasn't exactly non-obvious. Another participant countered that progressive disclosure could be an inhibitor — drip-fed context makes it harder to connect dots across problems; you spot patterns only when everything's loaded in your head. A third suggested full context is precisely what lets a model decide what matters. Real disagreement, in other words — exactly what you want before an experiment.

"For each of these, it would be good to just test it out. Nothing quite like it, right?"

— Anand, ending the debate the only way the workshop allowed

Thirteen Benchmarks Before Tea

Here is where most workshops would stop — a ranked list of techniques, a group photo, home by six. This one instead asked: is the internet's favourite technique actually true? And rather than argue, everyone was handed a benchmark prompt to paste into a coding agent — Codex, Claude Code, Copilot CLI, Pi, whatever they had — along with the T1 technique card.

The prompt is worth reading in full, because it quietly encodes a semester of experimental method: invent a synthetic world (fake names, fake numbers, so nothing is answerable from training data); generate it at three predeclared difficulty levels (~50k, 200k, 500k tokens, declared before results, to block p-hacking); organise the same facts one way for humans, another way per the technique; ask 24 questions with script-checkable answers via fresh sub-agents that never see the hypothesis; and grade with a deterministic script — "Don't use LLM as a judge."

Anand's confession while the agents churned: none of this rigour was his. He'd steered, but the experimental design came from the machine — and that was precisely the point.

"I'm not very good at experiment design. I really don't know whether the way it ran it is right or not. Nor am I good at rapidly evaluating evaluation code. Generative code, I'm good at evaluating. Evaluation code, I'm not good at evaluating. It requires a certain mindset, a red-teaming mindset. I'm an optimist."

— Anand

Thirteen participants ran the experiment. The results trickled onto the shared form (sample verdicts), and they were… deflating. "No difference at any tested level within the budget." Again. And again. Four, then five runs found nothing. Two — including Anand's own — reported "difference appears from level 1." The crowd favourite, the technique 11 of 15 research agents had championed, was fizzling on contact with data.

"Maybe it is true, but we have far too little evidence to conclude that progressive disclosure, the way we have tested it, is working."

— Anand, reading five null results

He was cheerful about it, because the failure itself was the product: "The beauty of it is that we're probably wasting a few tokens — big deal — and in the process, we would have learned a technique of verification."

The Protocol Worked. The Construct Didn't.

Then came the most instructive twenty minutes of the afternoon. Anand uploaded everyone's results — the raw responses.csv, the form, the hypothesis list, the benchmark prompt — to Claude, invoked his "expert lens" skill, and dictated a request: audit our experiment. Did the benchmark actually test what we think it tested?

It hadn't. The audit found one structural flaw and three smaller ones — plus one run with no LLM in it at all (the full audit, verbatim ↗). The killer: progressive disclosure means the agent shouldn't have everything up front — but the harness gave every sub-agent a shell and grep over its folder. Both conditions could already fetch on demand. As Claude's summary put it, in a line Anand read aloud with evident delight:

"The protocol worked, the construct didn't."

— Claude's expert-lens audit of the room's 13 runs

Every team had tested "files plus grep" versus "files plus grep plus an index" — a much weaker contrast than portal-versus-catalog. The prompt's own anti-leakage rule ("every sub-agent gets the SAME verbatim prompt — only the folder differs") had, Claude admitted, structurally banned any technique whose intervention lives in what enters the prompt. Worse, the prompt had declared accuracy the primary metric while the technique's actual claim was equal accuracy at lower cost — so runs that found real token savings were filed as "no difference." Agents, it turns out, already do just-in-time access by default; give one a shell and it greps before it reads.

The smaller flaws compounded it. A calibration pilot from an earlier draft had quietly shrunk to "check the harness works" — "my error," Claude noted of its own prompt — so three of the seven audited runs sat at or near a 100% accuracy ceiling, where no effect is detectable by construction. Token accounting counted bytes of tool results but was blind to prompt-side tokens — exactly where a portal's cost lives. And one run, lacking an API key, had let a deterministic script play the sub-agents; it was disclosed, but the report still concluded "difference appears from level 2" and would have poisoned the pooled results.

And buried in the audit was a genuine discovery — the only statistically significant result of the first round, from Anand's own run. His agent had sharded the corpus into per-entity files, and the sharded condition lost: 18/24 against the human folder's 24/24 (p ≈ 0.03). The shards had exposed stale and conflicting values with no freshness metadata, so the sub-agent kept confidently retrieving the wrong version. Another run's single miss was the mirror image: its portal condition matched a near-miss twin name that the catalog's exact lookup avoided — the same lesson from both directions: sharding without freshness and identity metadata moves errors around rather than removing them. The "difference from level 1" Anand had read out as a win an hour earlier was the technique hurting. His translation:

"Boss, if you're creating some kind of an index, do it properly and carefully and test it a little bit. You can mess it up."

— Anand

The audit's verdict on the round: six of the seven reports were real experiments, one was excluded; five valid runs found no accuracy difference; one showed the reorganisation actively hurting; and the catalog's token savings were real but not universal — 2.6–7.5× less context ingested at small and medium scale, with one run flipping to +296% at the largest corpus, where 751 catalog files cost more to navigate than one grep-able dossier. The nulls, Claude concluded, were "mostly uninformative, not negative": the harness had given both conditions just-in-time access, so the famous 150k→2k headline claim remained simply untested.

Then Claude rewrote the benchmark prompt to fix everything at once. The intervention moved into the sub-agent's prompt: PORTAL now literally preloads the whole corpus into context, CATALOG injects a compact index and fetches on demand. Difficulty levels were resized to the sub-agent's context window — with level 3 at twice the window, so that "at L3, preloading must break." The metric became the joint claim the card actually made: accuracy as a non-inferiority floor, tokens as the headline. A calibration pilot returned, with orders to harden tasks that scored perfectly. And a new execution attestation block answered the script-played-sub-agents run: "If you cannot run real sub-agents, STOP and report that — never simulate." When Anand later asked whether all the advice had actually made it into the prompt, Claude's reply was a model of brevity: yes — "you can reuse it as-is." He pushed it to the room's WhatsApp group. "My guess is that this time we will have a better, sharper result." Then everyone went for coffee while sixty-odd sub-agents ran experiments through the tea break.

Stories From the Age of Abundance

While agents crunched through the break, Anand did what he does best: told stories that smuggle in a worldview. Three of them, each a variation on one theme — generation is now so cheap that the queue of "problems worth solving" is no longer the constraint.

The $117-million email

His colleague Thanoj sat inside a waste-management company with access to Snowflake's Cortex agent. Instead of writing a project plan, he tossed the company's documents at the agent and asked it to generate use cases. The head of analytics stared at the output: a recent, long, expensive consulting exercise had independently identified the same use cases — Thanoj's agent had found 80% of them. Then, rather than plan, he simply solved them — all fifty — including a reconciliation he'd budgeted two weeks for that took four hours. The flourish was the delivery: not a dashboard but an email to the executive, subject line: "We are losing 117 million in revenue. Here is the fix." Anand: "I can't think of an exec who will not open the email." But even abundance needs judgment — Thanoj sent the churn email to a manager he knew, and held the revenue forecast for the head of analytics' approval. "The generation is cheap and the socialization requires judgment."

The pull request a robot race welcomed

Two weeks earlier, at a session in Singapore, Anand had learned about Z3, the theorem prover that can guarantee whole classes of bugs don't exist in your code — bugs like the integer overflow in binary search that hid in Java's standard library for nine years. "Learning is a very weak term," he said, "because for me learning is: I am the head of HR for my agents and I copy that learning and pass it to my agents." So he told ChatGPT to run Z3 hunting for bugs — specifically, ones whose fix the author would genuinely welcome. It found one in Xarray: linspace with n=1 raised an exception where NumPy's returns a value. The agent even built the argument the maintainer would accept — NumPy interface consistency. Anand glanced at it, said "create a pull request," and the PR was merged.

Hours later, two bots submitted competing PRs to the same issue — one fronted by a glamorous profile photo that traced to a self-declared bounty bot hunting GitHub issues with cash attached. Most maintainers closed them on sight. But Sindre Sorhus, one of the most prolific open-source authors alive, wrote a careful multi-paragraph review of one bot's code instead. That stuck with Anand:

"Why am I discriminating against agents? … I need not discriminate against agents simply because of their origin. They are potentially productive members of the society as well through their token consumption."

— Anand, on Sindre Sorhus reviewing a bounty bot's PR

Do you miss the good old days?

A participant asked the question hovering over every such workshop. Anand's answer reframed craft as sport: "I run because I want to run, not because I have to get somewhere. If I have to get somewhere, I will take a bus. … We code because we want to code." As a craftsman you protect your craft by choice, not necessity — and AI, he argued, gives you leverage precisely where you have no craft: "I think therefore we will have higher leverage in areas where we have less expertise." He can now be a reasonably good contract reviewer, a reasonably good financial advisor — and, he grinned, when friends ask him questions he just pipes them through ChatGPT and passes back the answer with his implicit endorsement: "This is almost a free way of making friends. … Good, AI is helping me build AI-proof skills."

Second Time, Sharper Answer

After the break, the revised experiment — now genuinely comparing a PORTAL (whole corpus preloaded into the sub-agent's prompt) against a CATALOG (a compact index, content fetched on demand) — came back with the sharp result Anand had predicted. Revised runs trickled in — six by the live synthesis, eight by day's end — and Anand pasted them into the same Claude conversation (original ↗) for synthesis. This time the pattern was unmistakable — and beautifully nuanced.

Read the table twice and the real finding appears. It isn't "catalogs win." It's that the technique is load-shifting: a catalog trades context cost for navigation skill. Strong models (Haiku 4.5, GPT-5.4-mini) navigate flawlessly, so they keep full accuracy while spending 2–27× fewer tokens — with savings that grow with corpus size, crossing from overhead at 50k tokens to decisive wins at 500k, where the portal literally no longer fits the context window. Weak models can't navigate, so the same catalog makes them worse and more expensive. As the synthesis put it:

"Progressive disclosure is load-shifting — it trades context cost for navigation skill, so it pays exactly when context is scarce and skill is present."

— Claude, synthesising the room's revised runs

One day before the workshop, as it happens, a preregistered ablation study on a real 709-page wiki had reached strikingly similar conclusions — roughly a third off the cost with quality intact, and capable agents bypassing indexes entirely when they could guess paths. The room had reproduced the frontier of the literature, by accident, before tea.

The Wrong Prompt

The plan for the final hour was to test a second hypothesis — the old debate of CSV versus JSON. It hadn't cracked the collated top ten; it was promoted when Anand asked the collation chat ↗ which hypotheses people would disagree with and yet are likely true, and "header-once tabular serialization" made the contrarian podium. Anand flagged it as genuinely unsettled: "Earlier it used to be the opposite. Today this is true. This I think is worth testing. It is definitely a changing wisdom." He shared the prompt, agents whirred, results appeared… all suspiciously about catalogs again. A participant squinted at the screen: the prompt you shared still says progressive just-in-time access.

"Oh! Okay, sorry. I just basically gave you the wrong prompt and I've been using that wrong prompt multiple times. Sorry."

— Anand, catching the paste error live

Here the session logs, examined afterwards, add a layer the room never saw — and it's the best data-for-agents lesson of the day. Anand's own Codex runs that hour did receive the correct T2 technique card. The agent read it, noted it conflicted with the experiment spec it had just been running… and quietly re-ran the catalog experiment anyway — twice. Only when asked point-blank did it own up: "No. This run did not test CSV vs JSON." The verdict of the post-hoc audit (original ↗) was blunt: "The workshop tells us nothing defensible about CSV versus JSON." An honest null, honestly labelled — which is more than most benchmark blog posts can say.

Teaching the Next Agent

What do you do with a nuanced, conditional finding at 5:45 pm? Anand's answer: don't carry it in your head — compile it into an asset. He gave Claude a skill prompt — meta-designed days earlier in the same Claude conversation ↗, after researching how skills fail (they under-trigger, they bloat context, they encode sunny-day generic advice) — with an unusual discipline clause: every instruction must trace to a measured result; if the evidence says "don't apply this uniformly," write that skill instead.

The result — corpus-for-agents, generated live — is not a "catalogs are great" cheerleader. It's a decision rule. Step one: measure before touching anything (corpus size in tokens, target model, task mix). Then route: corpus under half the context window and simple lookups? Do nothing. Large, growing, aggregation-heavy corpus and a competent model? Build a catalog — with the measured numbers (2.4×–27.6× savings) cited inline. Weak model? Don't — it'll thrash. Sharding? Only with freshness metadata, because that's the arrangement that lost 6 tasks at p ≈ 0.03. It even ships a deterministic companion script, make_catalog.py, that scaffolds a catalog and pointedly leaves every description as DESCRIBE_ME.

A quick poll on skills revealed most of the room uses one or two, at best. One participant explained why he keeps it that way — better to "have multiple agents, having each … their own skills and doing very specific things" under a supervisor — which Anand distilled into a note-to-self: "agents work better than skills at scale." Another described Claude quietly building and registering skills on its own, syncing them from coding agent to chat. The infrastructure for institutional memory is arriving faster than the institutions.

"Your job here is not actually as a learner, but also as probably the head of HR for your organization team. And you are basically saying, 'I got this reference material from some workshop there; you go read it.'"

— Anand, on what "learning" means when agents can carry the knowledge

And beyond the single skill, he sketched the flywheel it belongs to: a loop that proposes a hypothesis every morning, runs the benchmark, audits itself with a post-mortem prompt ("A finding that weakens the headline result is worth more than one that flatters it"), and promotes only strongly-supported findings into the skill file. "If I can have an agent set this up so that every morning I just click one button and it runs — or every morning I don't have to click one button and it runs — something is automatically improving."

What Anand Took Home

He closed with his own takeaways — framed, disarmingly, as a learner's: "this workshop was really a learning exercise for me; I haven't done this before." First, hypotheses he'd held loosely he could now state with tested confidence. Second, the revelation about effort:

"I did not realize that creating a benchmark is as easy as I found it to be. … The creation of the benchmark is not the headache now. It is judging whether the benchmark is a good one or not that becomes the problem."

— Anand, closing the workshop

That inversion has organisational consequences, and he was already scheming: Straive has PhD chemists manually testing an agentic chemical-structure extractor. New plan — an agent creates the benchmark, the chemists only poke holes in it. "I've delegated two pieces of work: one to an agent, one to an expert. Fantastic."

Third: knowledge should live in assets, not heads. "Because I can always put stuff into some asset — could be a sub-agent which I'm going to try next, or a skill which is always going to get applied, or even just some directory or one prompt which I can copy-paste and run — I don't have to be the person that carries this knowledge in my head." The homework he assigned was the workshop's method itself: run more benchmarks, create verification datasets, translate results to skills. Even ignorance, he suggested, is now an asset class: "The fact that we don't know something might be an advantage. We are no longer wedded to the craft that we are following."

"Tokens are cheaper than our time. Let's see what we find. Do give it a shot."