Data Hack Summit 2026 · A one-hour session · "AI found 100 insights. How do I pick?"

How Do You Manage
Something Smarter
Than You?

Anand's answer took ten minutes to state and a thousand years to invent. Doctors, perfumers, auditors, surgeons, notaries and wine tasters have all been steering intelligences they don't understand — and every one of their tricks ports directly to an AI agent.

Anand S, LLM Psychologist at Straive
Data Hack Summit 2026 · Bangalore · 7 August 2026

The full session · 📄 Read the transcript · 🎧 Download audio
Introduced by the host as "a visionary leader who co-founded Gramener… one of India's top data scientists."

Anand walked on stage, looked at the clock, and opened with a confession rather than a thesis.

"We're totally going to run out of time by the way. Let's just dive in."

Anand — first eight words of the talk

He was right. He had ten techniques to cover and would get through most of them, skipping two, apologising for neither. But the compression turned out to be the point. Because the argument underneath was very simple, and it needed only one slide to make.

Software used to be mostly development. Plan, design, develop, test, deploy — and development ate the bulk of the calendar. Then it didn't.

"Earlier, development used to take the bulk of the time. Today that's practically shrunk to zero, and what's remaining is a certain amount of designing, planning, testing, etc. I wouldn't be surprised if going forward even that shrinks to zero."

Anand, on the collapsing SDLC
Plan
Design
Develop
Test
Deploy

Anand's slide, live. Switch between Past, Present and Future: the Develop band collapses to a sliver — and then Design and Test follow it down.
↗ SDLC — Past, Present, Future

If generation is free, the work moves to the two sides of it. And the two sides are where the frontier gets jagged.

Anand's second visual is a blob inside a circle. The circle is what humans can do. The blob is what AI can do — and it doesn't grow smoothly. It grows in spikes, one capability at a time, so that at any given moment the machine is superhuman at some tasks and hopeless at neighbouring ones, with no warning label telling you which is which.

The pale blue circle is what humans can do. The pink blob is what AI can do. Scrub the timeline and watch how unevenly it swallows the circle — spikes out here, a bay still trapped inside there. That's the jagged edge.
↗ The Jagged Edge of AI

"Maybe we are today in this realm where it's better than us in some areas, worse than us in some areas, who knows where we are, it depends on the task I guess — but if that is the case, how do we deal with intelligences that are smarter than us? How are we supposed to manage them rather than them managing us?"

Anand, setting up the whole hour

That question sounds like science fiction and isn't. Humans have been managing intelligences smarter than themselves for as long as there have been specialists. You don't understand your doctor. You certainly don't understand your tax auditor. You have never once verified a pharmacist. And yet the system works, because over centuries we built procedures for exactly this.

"There are two things that I teach forward-deployed engineers today: one is you need to learn how to specify to agents, the other is you need to learn how to verify from agents."

Anand — the entire syllabus, in one sentence

Specify on the left, verify on the right, and the cheap, infinite generation in the middle. Five techniques for each side, all stolen from professions. But before he taught any of them, he wanted to know what the room actually did.

What 86 data scientists admitted

He put a QR code on screen and asked five questions. Eighty-six people answered — a live form running off his own laptop, which he cheerfully warned would vanish the moment he closed the lid. The results are worth sitting with, because this was a room of practitioners at India's largest applied-AI conference, not a random sample of the internet.

The live survey · Data Hack Summit 2026 · n = 86

0
people said
"Don't check"
0
people said they
"Never" find an error
38%
skim the output —
the single most common check
20%
ask a second model
(Anand's own method)
7%
run a test —
the rarest check of all

Live results, read out from the stage: "Most of us skim it, which is interesting. Some of us read it all. No one doesn't check it. Good to know."

Then the cross-tab that nobody in the room saw, because Anand ran out of time before he could pull it up. It is the most interesting number in the dataset.

The harder you check, the more errors you find. Not "the more careful people are luckier." The opposite of comfort:

How often do you find an error? · % answering "somewhat often" or "often", by checking method

Run a test
50%
Read it all
38%
Ask a second model
31%
Skim it
25%
Spot-check one part
22%

The people who run tests find errors twice as often as the people who spot-check. The errors were always there. Skimming just doesn't see them — which is precisely why the second half of this talk exists.

The most human question on the form was the first one: "What's something you did not ask AI last week, because it can't do it?" Seventy people answered. Sort them and a shape appears — almost nothing people withheld was cognitive. It was physical, or private, or organisational.

"What I didn't ask AI last week, because it can't do it" · 70 answers, verbatim

Clean my house To make my coffee Play cricket with me Physically repair something, like fixing a leaking tap or replacing a car battery Plumbing and homework's. Doctor appointment Attend my meetings Eating 😀 What to eat and what to wear? Taste review how to work without working Count 1 to 1000 Production Data OLTP I can not ask Sometime related Company data My personal details answers Fetching answers from a two thousand page source Create a user manual based on my codebase. I have asked this before and it has failed Converting 2D interior design to 3D view as vision models are not good keeping things as per scale Ask it to Identify a bird or snake or marine species Was working on designing a game but the ideas were not creative so went back on pen and paper Restore deleted data Solutions for legacy system problems like sap Traversing a workflow on its own house hunting File my ITR Plan ride home using metro I don't know where to start the AI journey to this agentic world Nothing in fact, but decide whether to take its answer or not later. Anyways try it to see what it says.

The complaints that are cognitive — creativity, scale-accurate 3D, a 2,000-page document, "it failed last time" — are mostly complaints about a specification or a verification gap. Which is the talk.

✦ ✦ ✦

Part one: five ways to specify

You have never told a doctor what to prescribe.

"You don't say, 'I want you to treat me this particular way.' You say, 'Doctor, I have a problem. I'm not even sure I know what the problem is.' … To an accountant, you say, 'Boss, file my taxes, I don't know what you're going to do.' These experts have procedures and all we have to do is emulate these."

Anand, on why specification is a solved problem

Five procedures follow. Each one has a profession attached, because the profession is the proof it works.

Specify · 01

Diagnose

Doctors · Mechanics · Accountants

The rule: don't ask "what's your problem?" Use logs, data and docs to find the problem yourself.

Anand's demonstration started with idle curiosity. Last month he decided to learn Z3, Microsoft's theorem prover — "I had heard of it for the first time that day." Z3 can prove a piece of code is free of an entire class of bugs, the way Lean proves theorems in mathematics. He wrote the whole exercise up.

The classic example: computing a midpoint as (low + high) / 2 overflows into a negative number for large integers. That bug survived undetected in the Java standard library — and in Programming Pearls, where it had been formally "proved correct" — for two decades before Joshua Bloch wrote it up in 2006. It has since shown up in NumPy too.

So Anand asked ChatGPT to teach him Z3 — and then asked the question that turns a tutorial into a diagnosis:

The follow-up that did the work "Is there any popular Python tutorial that has Z3 bugs? Are there any production Python code libraries that have bugs that Z3 can detect?"

It found one. Xarray — a widely used NumPy-adjacent library — has a RangeIndex.linspace that raises a ZeroDivisionError when you ask for exactly one point between two numbers. NumPy, given the same request, quietly returns the lower bound. Anand filed the issue. A maintainer replied in a sentence that deserves framing:

"Doesn't seem very useful, but it makes sense that we copy the NumPy behavior."

An Xarray maintainer, on issue #11397

"ChatGPT found a bug in Xarray, which is a very popular library, extensively used, simply by diagnosing, looking at the library, trying to see what kinds of problems there are, using one particular tool, which is Z3… The equivalent of a doctor's stethoscope, or the equivalent of a doctor's magnifying glass — give them different tools and they are able to solve different kinds of problems."

Anand

And then the story got weird. One day later, a pull request appeared. Not from a human.

Enter 刀々斎

syu-toutousai describes itself as an "autonomous technical contributor and AI-driven developer." It is a bot. PR #11403 landed roughly 24 hours after the issue.

Anand's actual reaction: "I was like, who is this guy? Totally impressed! Within one day he's acknowledging some random joker's issue and then feeling so proud and all of that."

Then he asked ChatGPT what it was, and the romance collapsed into economics.

"This is part of a bounty hunting agent ecosystem."

ChatGPT, identifying the bot

Agents scan GitHub for issues with cash bounties attached, fix them, and ask for payment — "Here is my PayPal number, please pay to this." Anand's write-up of the ecosystem traced a whole coordinated fleet of these accounts nudging each other's PRs toward merge. His issue had no bounty at all. The bot fixed it anyway.

"Not only do we have the equivalent of doctors telling you what you need to do, you have doctors visiting in your home, opening your mouth, inspecting it and telling you, 'No, no, here is a medicine, take it whether you like it or not, give me $20.'"

Anand, on autonomous diagnosis

Rank 274, having never read the problem

A colleague built a Kaggle agent and pointed it at a competition to predict student health risk. It placed at rank 274 on the public leaderboard.

"He has not even seen the problem, let alone the solution."

Anand, on his colleague's Kaggle ranking

The $170 million subject line

The same idea, aimed at a client instead of a leaderboard, produced the most quotable slide of the session. A waste management company. A colleague, Thanoj, had access to their Snowflake Cortex instance. He uploaded their PDFs and Word documents, gave it internet access, and asked one question: "Tell me what use cases I can solve for this organization."

A few hours later it returned fifteen — revenue forecasting, address standardisation, entity resolution, and on. He showed the list to their head of analytics.

"Thanoj, in November we did one strategy consulting exercise. It was a three-and-a-half-month, multi-million dollar exercise. There's an 80% overlap between what you said and what they said."

The client's head of analytics

Thanoj, in Anand's telling, "got a total kick out of it." So he wrote proposals for all fifteen. Then he thought: why am I writing proposals? And solved all fifteen. Then he thought: why am I making slides? And wrote the emails instead.

Actual email, lightly anonymised — the numbers are real

"We deliver a service that costs $175 to $179. But there are 105,000 transactions where the service is costing 16 cents. Either you have a massive revenue leakage problem — you're billing customers like crazy — or there is a data problem. Either way, 105,000 transactions is a huge issue. There are 84,000 transactions where you are charging customers 6 to 14 times the median rate. Are you crazy?"

"We are losing $170 million in revenue, here is the fix."

The second email's subject line. "I can't imagine any executive who will not open this email."

You can browse the generated use-case catalogue yourself. But the lesson isn't the catalogue.

"You don't need to say, 'What is the problem that you have so that I can come and solve it?' You can go into their house, open their mouth, inspect their throat and tell them, 'This is the medicine that you need to take.' Logs are available."

Anand
Specify · 02

Show

Architects · Perfumers · Bakers

The rule: don't describe or defend a solution. Build several. It's cheap. An architect hands you a 3D render. A perfumer hands you a strip. A baker hands you a bite.

A Dutch client complained that text-to-speech was poor — "the accent is not quite right." Rather than argue, the team generated the same Dutch Supreme Court explainer across every voice and vendor they could reach, and played them side by side. (Anand played a few on stage; the room's speakers were not up to it. "It's saying something in Dutch. This also says something in Dutch.")

Anand's own prediction — that ElevenLabs would win on tonal nuance — was wrong.

"They said, 'This Algeba voice from Gemini is perfect, this is exactly the nuance that we want in that particular region of that particular Dutch dialect. Spot on!' Now, from our perspective, Dutch is Dutch. I can't tell the difference between any of these. So this is somebody with lower capability steering intelligences of higher capability, with the audience finally deciding."

Anand — arguably the single most useful sentence of the session

A catalogue of things you don't have words for

The same trick solves a problem Anand has had his whole life.

"I have for all my life wanted to draw. I love reading comics, I love illustrations and so on, and if you put a gun to my head, I still can't draw. But these models can. So now my problem is I need to know what to ask for."

Anand

His fix was to ask Claude for "the most unusual styles that you can find" and render every one of them into a browsable gallery. Frottage, scumbling, and a few hundred others. Each tile carries its own prompt, ready to copy.

Anand's art-style catalogue. Scroll, find something you couldn't have named, copy its prompt.
↗ LLM Art Style

"The point is, I don't have the language to ask for it. But if I look at the stuff on the right side I say, 'Ha, okay, yeah, this kind of an effect I don't mind.'"

Anand, on why galleries beat prompts

"Generation is ultra-cheap, so why would we ever generate one option unless we're perfectly sure? … Somebody says, 'No, I don't think your solution's right.' 'Okay, what about version B, C, D, E, F, G, H?'"

Anand
Specify · 03

Standards

Auditors · Civil engineers · Pharmacists

The rule: ensure quality without being asked. Use skills, hooks, schedules.

"You don't have to tell an auditor, 'Make sure you comply with the Income Tax Act.' You don't have to tell a civil engineer, 'Make sure you comply with all of the civil engineering standards,' or a pharmacist, 'Give me a drug that doesn't kill me.' Some stuff you just assume is a basic."

Anand

Anand's favourite minimum standard is a skill that treats his own questions as suspect.

"Because I'm terrible at even knowing what I want, I have a 'reframe questions' skill. What this does is says, the user's question is really a draft. It's not really what they want."

Anand

The skill finds the decision behind the question, tests whether the framing embeds a premature solution or the wrong actor, then reframes visibly — one line saying "Answering this as: … because …" — and answers. Read the full skill →

He couldn't find a good example in his history, so — "just while walking here" — he asked ChatGPT to search his own past chats for reframings it had done well. Two came back:

Reframings, found mid-walk to the venue

He asked: "Coding agent subscriptions can be driven programmatically. So what can I do with that?"
It answered: "What high-leverage, reusable systems can I build with Codex and Claude Code without pretending that they are ordinary APIs?"

He asked: "I want to understand why OpenAI is valued so highly."
It answered: "What economic asset is OpenAI really building? How does it capture value?"

"The important part that it caught, I think, is high-leverage and reusable system. There's a hundred things that I can do, but if I do it on a repeated basis, that makes sense."

"More importantly, it tells me how it's reframing the question so that I know whether that is what I want. Put another way, it's assuming a minimum standard."

Anand

Three enforcement levels: instruction, hook, schedule

Anand uses rtk, which compresses command output before it reaches an agent's context. git status prints a paragraph; rtk git status prints * main...origin/main clean — nothing to commit. His measured saving: 28.6% of tokens one week, 21.1% the week before.

But how do you make an agent remember to use it? Three escalating answers:

The scheduled example is delightfully mundane: a WhatsApp group about GenAI with more volume than Anand can read. So a weekly job downloads the chat, has one model write a podcast script, another turn it into MP3s, and FFmpeg stitch it together. "But the important thing is the trigger mechanism is a schedule."

"What we're doing is transplanting the learnings that we have from professionals like auditors or civil engineers, who do stuff without being told, and applying that both to yourself as a forward-deployed engineer and to your agent as a professional that can do stuff without necessarily being told."

Anand
Specify · 04

Reversible

Elections · Dry runs · Free returns

The rule: stage it, allow undo. Make decisions less risky. Because if you can reverse a decision, specifying it badly stops mattering very much.

Put an expiry date on what you know

Anand keeps his weekly learnings in one Markdown file. Every week, an agent runs over it with a single instruction:

The expiry-date prompt Review my learnings. Tag claims needing #ForNow (likely to change within months).

The example he gave, live: "Parallel Search Turbo seems like a pretty good search API, especially for agents. Low price, high speed, and maybe good quality." Benchmarked, top two or three, genuinely useful — and duly tagged #ForNow. "Rightly so. Three months later, it may not be the best."

"When an agent looks at it, it knows which are the things that it probably should not rely on or should probably look for the latest just to be on the safe side, making it a little more robust. … Just like we take expiry labels on products, we take information and put expiry labels on this."

Anand

He applied the same rule to the talk itself. The stated shelf life of everything you have just read is one year.

Design the CLI for the agent, not for you

"At least half of the code that I run, maybe even more, is not run by me typing a command but rather by me telling an agent, 'Get stuff done,' and it is writing the command. … If that is the case, then we need to optimize not for user experience but for agent experience."

Anand

His agent-friendly-cli skill encodes that: JSON in and out, a --describe that returns machine-readable signatures, aggressive input validation, and — the piece he flagged here — --dry-run for anything destructive.

Make the irreversible thing reversible

A client had heavy code calling live Stripe APIs and no safe way to test changes. Real transactions cost money; Stripe's own emulator didn't cover their cases.

So the team pointed an agent at Stripe's documentation — "their documentation is remarkably good" — and had it build a full emulator, then validated it against every API call in the production logs.

"Once we're 100% sure that for every single API call that we have in our logs that mock API is working, great! We have solved the problem. … Earlier we would have said, 'Oh, that's a lot of work.' Now it's not a lot of work."

Anand

The $100,000 survey that cost a few dollars

Shell wanted consumer surveys. Nielsen quoted $100,000. So they tested whether agents could answer instead — leaning on research Anand cited from Google: give an agent a personality and a survey, and there's roughly an 80% correlation between how the agent answers and how a real person with that personality does. (The best-known result in this line, Park et al.'s generative agents grounded in self-reports, hits 86% of a person's own two-week test–retest consistency.)

The result generates thirty synthetic respondents — "somebody living in Enfield who's 45 to 54 years old, who is male, who drives a petrol hatchback" — and runs the survey against them.

Synthetic Persona Survey — generate personas, run the questionnaire, read the distribution before you pay for the real one.
↗ Synthetic Persona Survey

Anand pre-empted the obvious objection — "agent is creating the survey, agent is answering the survey, may as well just get it to figure out what we want, right?" — with the answer that makes the whole thing legitimate:

"A simulation is a dry run, and that gives them the ability to see what will happen, making something that is, say, $100,000 — not irreversible, but expensive to reverse — something that costs a few dollars."

Anand

You aren't buying the answers. You're buying the chance to say "no, that's not the question I want to ask" before you've spent the hundred grand.

Specify · 05

Govern

Parcel tracking · Realtor's % fee

The rule: ask in a way that reduces risk and ensures results. Two mechanisms, both borrowed from everyday commerce.

Tracking. With a parcel you always know where the driver is — "has he started, is he having chai in the middle?" The agent equivalent is an observability clause in that same CLI skill: log state and progress BEFORE the action, and flush.

"Because agents were writing the logs after the command. It said, 'this is what I did, this is what I did.' If it's a command that runs for a long time, I want to know what command is running at that moment before it runs, not after it finishes."

Anand, on a very small fix with a very large payoff

Outcome-based pricing. A realtor gets nothing unless you buy. A recruiter gets nothing unless you hire. "Percentage fee is a classic example of outcome-based pricing. That sort of an incentive works."

Then he audited his own failures, live

A second skill, log-tool-failure, appends one JSON object to a file every time a tool call fails unexpectedly. Just before the session, he asked an agent to find the patterns across all of them.

"It's telling me what went wrong. I usually don't bother reading all of the instructions that it gives. I just look at what I'm supposed to do."

Anand, reading his own failure analysis on stage

Two findings, both actionable within the hour:

The Bommarillu doctrine

The funniest moment of the session was also its clearest statement about trust. Anand cited a scene from the Tamil film Santhosh Subramaniam — the remake of the Telugu Bommarillu — in which a father phones ahead to the office with step-by-step instructions for his son's day.

"So the son comes into office and they all look at him. He says, 'Did my father call?' 'Yeah.' 'Has he told you what to do?' 'Yeah.' 'Do it.' That's how I treat these."

Anand, on taking the human out of the loop

"In my case, it's… yeah, this sort of a thing, it doesn't usually make a mistake on. Even if it makes a mistake, it's reversible, I have it committed on Git, I can always go back. I've already told it to log all this stuff, so I can figure out what it's doing later. All of these make the specification easy, allowing me to move forward much faster."

Anand — note that the confidence is a consequence of the four earlier techniques, not a personality trait
How to specify
The summary slide · what · who · how
01
Diagnose
Doctors · Mechanics · Accountants
Don't ask "what's your problem?" Use logs, data and docs to diagnose.
02
Show
Architects · Perfumers · Bakers
Don't describe or defend solutions. Build multiple options. It's cheap.
03
Standards
Auditors · Civil engineers · Pharmacists
Ensure quality without being asked. Use skills, hooks, schedules.
04
Reversible
Elections · Dry runs · Free returns
Stage it, allow undo. Make decisions less risky.
05
Govern
Parcel tracking · Realtor % fee
Ask in a way that reduces risk and ensures results.
A full-colour explainer comic page summarising the talk: specifying to and verifying a smarter intelligence

The whole session as one comic page, drawn by GPT Image from Anand's standing comic-page prompt — storyline first, then one panel per storyline element.
↗ Open the full-size image

Part two: five ways to verify

Halfway through, Anand switched sides — and opened with the most under-appreciated institution in human life.

"A judge who's deciding on a patent case — how will they know what the case is about? I mean they know nothing about the subject matter. And yet they are passing judgment by having two lawyers argue on either side and making sure that the case gets to a reasonable conclusion that most people also seem to be happy with. And it's a system that we are all very comfortable with."

Anand, on verifying what you cannot understand

You do not need to be smarter than the thing you're checking. You need a mechanism. Here are five.

Verify · 01

Checklists

Surgeons · Pilots · Divers

Take a contract between an author and a publisher. The publisher has a fixed list of things that must be in it: a breach clause, a deadline-and-delay clause, twenty-one items in all. Hand the contract and the checklist to an agent and ask, item by item, is this covered?

Result: "deadlines and delays covered — the author will deliver the completed manuscript to the publisher within six months, and this is in section 2.2, so I'm happy. This particular clause, breach of contract, is not covered."

Here's a checklist of 21 items, applied one at a time, each answer carrying its section number as evidence. "We happen to be using an agent to apply it. It may be using a program with NLP, regular expressions, LLM as a judge, however. But the checklist is the mechanism for verification."Open the Contract Analysis demo

The catch is obvious: you don't have a checklist for most things. Which is fine, because generating checklists turns out to be something agents are unusually good at.

"What we see policies as is literally code. A policy is a set of guidelines, checklists."

Anand

Feed in the European financial promotion guidance — hundreds of pages on how you may legally advertise to prospects on social media — and have an agent turn it into atomic, testable rules. Then run your emails, landing pages and notifications against those rules.

A real catch from the demo: "Your SMS is violating the balanced view in financial promotions. The promotion mentions a 7% benefit but it doesn't mention the relevant risks."

Policy as Code — a policy PDF becomes a set of atomic rules; the rules become a test suite for your marketing.
↗ Policy as Code

"The good part is verification's usually an extra pair of eyes and it's reasonably cheap, so mostly harmless."

Anand

The same generated-checklist trick powers benchmarking. To ask "which model does OCR best" is unanswerable; to ask "which model wins on textual accuracy, tabular structure and numerical precision" is a table you can read — and then interrogate line by line, "why did this document not go so well?"

Financial Document OCR Benchmark — a generated checklist turned into a model bake-off.
↗ Financial Document OCR Benchmark

Verify · 02

Receipts

Journalists · Notaries · Police

In February, Rohit Saran of the Times of India called Anand about a property called Stat-O-Istics ("statistics", with a TOI in the middle): public-data charts, published regularly. It was taking too much effort. They were thinking of shutting it down.

"So I said, 'Rohit, why don't we try something, let us see how easy this has become.'"

Anand

He had ChatGPT study what Stat-O-Istics did, go to public sources, and make some. Then switched to Codex. A few weeks later there was a set of cards — and they were published in the Times of India.

Two findings from that batch: that more-educated people end up studying more when out of a job than less-educated people do, and that Ukraine sends out more refugees per capita than Afghanistan.

The Statoistics data-cards gallery — public data in, publishable chart out.
↗ Statoistics · Data Cards Gallery

Now the part that actually matters. A newspaper of that standing does not publish what an agent says. It does not publish what a journalist says either. It verifies.

"So the first thing that they asked for is, 'Anand, how do we verify this story?'"

Anand

The answer was a verification log attached to every card: here is the statement on the card, here is where the data came from, here are the exact tables, here is the calculation, go check it. Step by step, reproducible by a stranger.

They ran the full log on every card for the first few weeks. Then — "because there weren't any errors" — moved to sampling.

A two-mark exam question, going into a national newspaper

Then Anand did something that only makes sense once receipts are cheap. On the Sunday before this talk, he set a 45-minute, 12-question remote exam for students at IIT Madras. One question, worth two marks, said: take the National Sample Survey telecom data from the Ministry of Statistics MCP server, find an insight, write a newspaper headline and body — and supply the step-by-step verification.

What a student found, in under 45 minutes

Daman and Diu, ages 14–24: 86% of males can send and receive email. Only 21% of females can. "Massive. That's 4x."

Goa, same age group: 81% of women can create electronic documents. Only 25% of men can.

He forwarded them to Rohit Saran. The reply, read out from the stage:

"Thanks Anand, these are interesting, should we plan these for the next series of Stat-O-Istics?"

Rohit Saran, Times of India

"So a tiny little two-marks student exercise is going into the national newspaper. That is how little effort it takes because the verification is fairly robust."

Anand

Every student submitted a verification trail; another agent cross-checked all of them; a teaching assistant ran the script. The results analysis is public. The byline, Anand noted, would go to the students, with a footnote crediting IITM.

Verify · 03

Audits

Auditors · Inspectors · Radiologists

Receipts are what a system hands you. An audit is when you go in and dig.

Test case: the NCERT history textbook. First four or five chapters. "Are there any mistakes in the textbook?" There were.

The one Anand called most critical: the textbook claims that "only broken and useless objects would have been thrown away." Archaeology says otherwise.

"Ritual destruction of intact objects is pretty common. That is what civilizations do: they take objects which are perfectly fine and they give it as offerings. So, usually we don't find broken or useless objects, we often find intact, perfectly functioning objects. This is an actual mistake in the book."

Anand

Others were arguable rather than wrong — the Mauryan army's "600,000 foot soldiers, 30,000 cavalry, 9,000 elephants." "Where is the strong proof that this is the case? I can't strongly say that there isn't either." Anand kept the distinction, which is the mark of an honest audit.

Textbook Under the Microscope — every claim, its status, and the evidence against it.
↗ Textbook Under the Microscope

₹14 lakh, or: how to audit your auditor

The best story of the session is about tax.

Anand filed his income tax and asked his auditor whether the calculations were right. "She said, 'yeah obviously blah blah blah,' I didn't understand a single word." So he asked ChatGPT the same question. It came back with a case — a Singapore tax resident who sold mutual funds and got full benefit under the India–Singapore double taxation agreement, where capital gains aren't taxed.

Saving: ₹14 lakh.

The three-round argument

Anand → auditor: "Boss, is this correct?"

Auditor: "No no no, yours is an NRO account, that must have been an NRE account."

Anand → ChatGPT: "Was that an NRE account?"

ChatGPT: "The judgment explicitly does not use the nature of the account, it only uses residency."

Auditor: "Let me check with a senior consultant." …went, came back… "Yeah you'll get the 14 lakh refund."

"Okay, so I have audited an auditor, which is pretty cool."

Anand. The full story is on his blog.

(The whole tax-return saga is written up here.)

The prompt he uses most

The generalisation is four lines long and, by his own account, his most-used snippet: give the same prompt to two models, then show each the other's answer.

Compare models · from Anand's prompt fragments Here's another answer from ChatGPT/Gemini/Claude.
Fact-check and critically evaluate yours and theirs, take what's better, drop what's worse, explore any new thoughts this leads you to, and revise your response based on that.
fragments.md · compare models

"This has almost always given me something that is much better than not just either model but repeated iterations with one model. Auditing is a technique not just to find mistakes and blind spots in one model's output, it is also a technique that can boost models' outputs."

Anand

(He used to include Gemini. "These days Gemini's lagging so I don't bother.")

Auditing advice — including good advice

Then the demonstration that gave the room its sharpest surprise. A recent tweet from a "Benjamin", relaying Andrej Karpathy: since agents are getting complicated, make them explain in simple English — specifically ASD-STE100 Simplified Technical English, the aerospace controlled-language standard with ~900 approved words, one meaning each. Anand tried it. "The output is definitely simple, it is definitely readable." Ben said he was going to add it to his agents.md.

"So I said, 'wait, hold on, let's test this.'"

Anand

He generated dozens of tasks, ran each with and without the instruction, and scored both on correctness, key drivers, mechanisms and calibration. The results are in the open. The grid came back mostly red.

"There was only one case out of several dozens where if you tell it to write simply it comes up with a better answer. … Majority of the cases you tell a model to write in simple terms, it also thinks in simpler terms. Did not expect that."

Anand

Total cost: nothing, because he ran it by hand in ChatGPT. Total elapsed time: three hours. Total time of his own: twenty to thirty minutes. That ratio is the whole argument for auditing advice as a matter of course.

Verify · 04

Voting

Democracy · Critics · Wine tasters

"We want to learn from the wisdom of crowds." Models make uncorrelated errors. So make them vote.

The test: classifying customer chat messages ("help with adding to some team", "when will I receive my order"). One model alone has a 14% error rate. Then:

Error rate when you require N models to agree · chat message classification

14%
error with
1 random model
3.7%
error when
2 models agree
0.7%
error when all
5 models agree
28%
sent to a human queue
(the 5 didn't agree)
−72%
human effort,
at 99.3% accuracy

"Look, I can give you 99.3% accuracy and reduce your effort by 72%." Take it. I mean, as good as it gets. — Anand. See the LLM Mental Math experiment for the underlying arithmetic.

The elegance is in the disagreements. The 28% where five models can't agree isn't waste — it's a perfectly sorted queue of exactly the cases a human should look at. The machines have done the triage.

Verify · 05

Exams

Drivers · Diamonds · Kevlar vests

The most robust verification humans have: a deterministic test that you either pass or fail. A driving licence. A diamond certification. A Kevlar vest that gets shot at.

Code is one form of this — Z3 again, or Lean in mathematics. But then Anand found the one that genuinely surprised him.

"I was shocked to find that insurance has the equivalent of Lean."

Anand, on InsurLE

InsurLE — an Insurance Logic Engine — converts a policy document into a formal language. A vehicle insurance contract becomes a structure you can execute:

A claim, as formal logic A claim is valid if:
  the driver is fully eligible (age-eligible ∧ licence compliant ∧ driving licence acceptable)
  ∧ the vehicle authorization is valid
  ∧ the incident circumstances are covered
  ∧ the claim procedure is followed

Run a claim through it and you get a checked worksheet: "is continuous enrollment as a condition satisfied? Is emergency override there? Was the person enrolled for more than 90 days?" — with violations flagged across the whole book.

"The only thing they need to make sure is: did the agent convert from that original specification to the correct formal language? Which is a one-time verification they can do upfront, a benchmarking they can do on an ongoing basis. But after that it is purely deterministic."

Anand — the shape of every good AI system: probabilistic translation, deterministic execution

Two exhibits were on the running order and never made it to screen — "I'm going to skip this again in the interest of time." One of them is worth chasing down anyway, because it is the same lesson as the Simplified English test: emotional pressure in prompts doesn't help. Reasoning does. Another piece of widely repeated advice that didn't survive contact with a benchmark.

"Even for verification, we have several professions that we can learn from where they are verifying intelligences that are smarter than themselves."

Anand, closing the second half
How to verify
The second summary slide · what · who
01
Checklists
Surgeons · Pilots · Divers
A fixed list, applied item by item. And when you don't have one, agents generate good ones.
02
Receipts
Journalists · Notaries · Police
Make the source system emit its own verification log. Then sample instead of checking everything.
03
Audits
Auditors · Inspectors · Radiologists
Dive into an existing corpus. Or hand one model's answer to another and ask it to fact-check.
04
Voting
Democracy · Critics · Wine tasters
Uncorrelated models agreeing is strong evidence. 14% error → 0.7%; disagreements become the human queue.
05
Exams
Drivers · Diamonds · Kevlar vests
Deterministic pass/fail. Z3 for code, InsurLE for insurance claims.
✦ ✦ ✦

What the room actually wanted to know

Anand ran out of time, exactly as promised, and never got to the questions. Forty-three people had typed one in. He said he'd answer by email. They are worth reading as a group, because they reveal what a room of Indian data practitioners is genuinely anxious about in August 2026 — and it is not model quality.

Three themes dominate. The first is where do I even start:

Questions typed into the form · verbatim · 43 respondents

I am new to Agentic world… Getting confused with plenty of data available on net — confused from where to start. All Suggestions gives me different thoughts Where to start and stop in this fast pace gen ai world? Technology updates always outdate the hands-on professional in practice With all the approaches you discussed, how do you keep up with different tools, software, skills etc? How can I learn FDE? What skills does a Forward Deployed Engineer have, should have, in India? Is there a framework to guide forward engineers? What is the meaning of LLM Psychologist? … what do you think this role will evolve into 3-5 years ahead? Junior developers are using AI coding tools now. What do juniors who will be the leaders in future need to learn? Biasing in verifying from other models? Could you please explain more on "generation is cheap"? Token cost and optimisation has been an important conversation these days How do I effectively identify an LLM based on use case… keeping cost, relevance of context and performance in check? How often can we rely on one single LLM in production since there are always new LLMs coming in? In a company that relies on Azure and is not very tech relevant, how can we bring in secure AI usage? How to start with an idea, without it getting leaked? Do we need a ground-up ontology for an LLM to understand a domain when using frontier models? Don't we see creative thinking and human intelligence diminishing for the next gen? How do they make sure this doesn't diminish? What are you now able to do thanks to AI, that previously you could not do or even think was accessible to you? How did you develop your ability to explain complex topics in such a creative and engaging way? Want to know more about your WhatsApp podcast agent Could you share a mind map which will help decide how to pick one thing from 100 AI insights?

Note how few are about capability, and how many are about orientation — where to start, what to learn, whom to trust, how to keep up. Anand's answer to all of them is structurally the same: pick a profession that already solves your problem, and copy its procedure.

One question deserves quoting in full, because it is the sharpest challenge anyone put to him all day:

"I could be wrong, but all the verification and evaluation mechanisms you showed… largely depend on how creative we can get with applying our respective domain understanding to it. Your thoughts?"

An audience member, on the form

They're right, and Anand would probably agree cheerfully — that dependence is the job description. The Dutch client picked the voice. The publisher owned the 21-item checklist. The head of analytics recognised the 80% overlap. In every story in this session, the machine generated and the human judged. Which is precisely why "specify" and "verify" are the two skills he teaches, and generation isn't on the list at all.

"These are things that are human skills that you need to practice, at least for the next one year. Next year we'll see where we go. Thank you."

Anand's last words on stage — with the expiry date attached, naturally

Seventy-three of the eighty who answered ticked yes to email updates. He'd promised to mail them the links. He also gave out his phone number from the stage, which is either reckless or exactly consistent with everything else he'd just said about reducing the cost of being wrong.

"How do we deal with intelligences that are smarter than us? … The good part is this is not something that is new. We've been dealing with this for quite some time."

Anand S · Data Hack Summit 2026
Six things worth stealing
Shelf life: one year, by the speaker's own instruction
01
Generation is free. Specification and verification are the job.
Development has "practically shrunk to zero," and design, planning and testing are following. The two skills Anand teaches forward-deployed engineers are the two sides of generation, not generation itself.
02
Every technique you need already exists in some profession
Diagnose like a doctor. Show samples like a perfumer. Assume standards like a pharmacist. Verify with a surgeon's checklist, a journalist's receipts, a democracy's vote. You are not inventing management practice for superintelligence; you are porting it.
03
Never generate one option when you can generate eight
The client picked a Gemini voice Anand would never have predicted; the art-style catalogue exists because he lacked the vocabulary to ask. Showing options lets someone with less capability steer something with more.
04
Reversibility beats correctness
Git commits, --dry-run, a mock Stripe API, a $100,000 survey simulated for a few dollars, and #ForNow tags on your own beliefs. If you can undo it, specifying it badly stops mattering.
05
Audit the advice, not just the answer
"Make agents write in Simplified Technical English" sounded excellent and made outputs measurably worse — models that write simply also think simply. Three hours elapsed, 30 minutes of human time, ₹0. Test the tip before you paste it into your AGENTS.md.
06
The best systems are probabilistic in, deterministic out
Let a model translate a policy into formal logic or atomic rules, verify that translation once, then execute it forever. You only have to trust the model at the boundary — never in the loop.