# Transcript

**Anand**: [00:03] Okay, let's dive in. Could we just do a quick count because depending on the number of people we have, we might need to rearrange ourselves just to segregate the participants from everyone else. So, all the participants, lift your hands. 10? Okay. And when the people come in, we can organize. Not a problem. I request that we then group ourselves into two per table. Each of these tables has a chart on them. You can redistribute yourself, please. No less than two. So yeah, please. On the other side is the TV, but I’m not going to be—you don’t need to look at it at all. This is more for me, and to be frank, I should just turn it off, but occasionally you will need to turn… Not really required, but you can point… Yeah, I think it’s going to be wiser if I come here. Okay. And two per table, that's what we need. Okay. It might be easier if people from that table—if you like this chart, you can take that chart there, sitting-wise.

**Anand**: [02:04] Okay, cool. What we are going to talk about is **how AI has made what we thought was really easy a little harder and what we thought was really hard a little easier.** Specifically, it’s very easy now to create charts. You tell AI to create a chart; it’s going to create a chart. So, that’s not the issue. But the other thing we assumed last year, at least, was therefore the taste in creating charts and all of that is now going to be the human domain. That doesn't seem to be the case. That is also something that AI is eating up. It's able to create some pretty good charts. So, okay, taste is not what remains. So, what is going to remain? I'm not sure, but what I’m hoping is that what we discover in this workshop will last us one more year. Beyond that, we’ll have another dialogue and see what happens.

**Anand**: [03:07] But therefore, my current hypothesis is what lasts is—and I’ll go through this step-by-step—one, **figuring out the purpose. And what I mean by purpose is: what are we writing for whom?** Now, that is a choice. At some level, you could say AI could do that better than we can, but at some level, it's also truly a choice. I like vanilla ice cream; I don’t like chocolate ice cream. It’s not about right or not; it’s about choice. And in that case, it’s hard for AI to take that over; it’s hard for a human to take that over because there’s no necessary right answer. So, **framing the purpose—what do we want, not even how to get there—what we want or what we think others want could be one of those.** And choosing among the different trade-offs.

**Anand**: [04:02] Please just evenly distribute yourself, and if two of you could sit on this table at least, that would be very good. Yeah, and preferably no more than three, so we can just somewhat evenly distribute.

**Anand**: [04:24] Another thing that seems to be lasting, or might last, is **we are verifying whether this is right or not.** There are some things, purely simply because of physical limitations, AI is not able to verify. And finally, **taking responsibility for what we deliver as charts.** What I mean by taking responsibility—and we’ll talk about this a lot more—is at the moment, we don’t really know how to reward and punish AI. **We know how to reward and punish long-standing humans. Companies are a good example.** We’ve done that. In fact, rivers, ships—a river can hold assets in some countries. Temples can hold assets. Ships can be taken to court as a separate legal entity, not even as the owner of the ship. So, all of those sorts of things we have mechanisms for. Yeah, please just distribute yourself across the tables, maybe two or three people per table.

**Anand**: [05:23] So, if this is the case, then it kind of raises the question: at some point, we will probably legislate AI as well. Meaning we will hold AI accountable, either saying this AI can be given a bank account, it will be fined a certain amount, it can be destroyed, etc. Yeah, please, just any of these tables evenly.

**Anand**: [05:51] The question remains, until that happens, how do we as humans stay accountable for the AI that we're using? And in any case, we as humans will be using AI beyond its own accountability. So, if we therefore say that that last bit of ownership is about somebody who decides—basically, I can say yes or no, and I’m choosing to say yes, that is my responsibility. **If something goes wrong, I stay accountable for it. And if it needs to be corrected, I will correct it; I am that person.** That, as ownership, seems to be one of those that will last for many years. I’m not saying decades yet, but years for sure.

**Anand**: [06:33] So, what we're going to do in this dialogue is, mostly you discuss and I'm facilitating the discussion. You have in front of you, in each of these tables, a chart. I'm not yet telling you… Oh, yeah, yeah, no, no, thank you. I should give these cards… face down, please. But I’m not yet telling you… Does anyone else have any of these sheets of paper face down? No? Okay, please put them face down. I'm not yet telling you what the charts are, who the audience is, what the purpose is, which of these are AI-generated, how much of which is AI-generated, etc. And I'm going to ask you a very unfair question, which is: **which of these do you think is the best chart?** Please wander around the table, take a look, no more than two minutes. The idea is, “Ah, I take a look at it, I understand it,” whatever. **“I think this is the best chart,” and you will be writing your name and putting your post-it note with your name against it.** And then we will go on to explore what it’s for, whether for that audience this is the right chart. Sorry, if you could just distribute yourself somewhat equally across the tables.

[08:14] [Background noise of participants moving and talking as they look at the charts]

**Unsure**: [08:29] Oh, the inauguration just finished, so I think there’s going to be a bit of a rush here in the future. Okay, there’s one chair here in the back if somebody wants to sit. But yeah, please associate yourself with some table at random.

**Anand**: [08:58] Oh, shit. Okay. Quick note, possibly the second-best talk of the day is happening at the big thing. Rohit Saran’s giving a talk. You absolutely should be at Rohit Saran's talk. I won't feel bad. In fact, I'm feeling bad that I'm not able to attend it. I hate all the other organizers. But strictly speaking, you will get more out of this talk than this session. But since you kind of… Okay, you don’t have a choice. Please associate yourself with some table in spirit, whoever is…

[09:32] [Laughter and background chatter as participants settle in]

**Rasagy**: [09:44] We’ll probably say no for other people and let them feel bad. Okay. If you don’t mind, people coming in…

**Anand**: [09:51] I don't mind, but they should be in Rohit's talk. Anyway, we let the people decide. Yeah, no, please, sorry organizers, please tell people how they should… However, if people have arrived, they would love to not be overwhelmed because there’s no capacity. Okay, fine. Now, just everyone please associate yourself with some table. Wherever you're sitting, doesn't matter.

**Anand**: [10:16] Okay, so given this, I should probably do a quick recap of everything and restart. And it’s okay, we are 15 minutes late, but I will end on time.

**Anand**: [10:32] Okay, here is the one-minute summary. **AI is creating stuff like crazy. So, what the heck are we supposed to do as a result?** Last year I said maybe it is taste—knowing how to create good stuff—but AI is able to create good stuff now as well. So, the question then becomes: what should we do? I don’t know, and my guess is that **we will be the people who will decide what gets published because we decide what people want, we decide what’s more important, and we decide to take responsibility.** And what responsibility means is saying that if you want a neck to catch, I am that neck, and I will make sure that stuff gets corrected if I’ve totally screwed up or if my AI totally screwed up. That broadly seems to be what will last a year, at least. Beyond that, we’ll see.

**Anand**: [11:26] So, what we're going to do is discuss how we go about building that, and we start now with a small experiment. Each of you are to please associate yourself with some table, wherever you're sitting. And against each of these, without knowing what the chart is for, who it is made for, what the purpose is, etc., here are six charts. Some fully AI-generated, some fully human-generated, some partly AI-generated. And I’ll tell you all of that at the end; that’s okay.

**Anand**: [11:59] So, make this unfair decision. **I want each of you to take a post-it note, write your name on it, and put it against the best chart.** You can go around and see all the charts. Yeah, see all the charts, go around, and put your picture against the best. Put your name on a post-it against the best.

[12:28] [Background chatter and movement as participants walk around the tables looking at the charts]

**Anand**: [12:36] Inside, outside, both are okay. If you need to turn the chart around and all, okay. Don't take too much time, just… think of yourself as a very busy editor of charts. 100 people have given you AI-generated charts. You have exactly two minutes to decide. However you're going to pick, just pick, move on. Just put your name on whatever you think is the best chart. Just one chart. Pick one chart and put your name against it. You'll be moving it around, you'll be adding other stuff to it.

[13:58] [Continued background noise, rustling paper, and muffled conversations]

**Anand**: [14:09] Keep moving, keep moving. Just pick the next one. If you want to go back, put your name against something, go back, put your name against it. Just stick your post-it on something, but just make sure you see—see all six before you make your pick.

[14:44] [Laughter and chatter continues]

**Anand**: [15:08] Okay, it's time to wrap up. Whoever's put your name, please take your seats. Please start taking your seats. Just write your name against one chart. Okay, time to take your seats, please. Yeah, please take a seat.

**Anand**: [16:07] Great. Okay, question to whoever feels like answering: **why did you pick the chart you picked?**

**Unsure**: [16:16] It looked the most cleanest.

**Unsure**: [16:19] Easiest to understand.

**Unsure**: [16:21] It was very clear.

**Unsure**: [16:24] Area of interest.

**Unsure**: [16:27] It gave context. There was some context or explanation.

**Unsure**: [16:34] Wasn't too complex, with too much data.

**Unsure**: [16:38] It didn't have an obvious mistake.

**Unsure**: [16:41] Didn't need too much context.

**Unsure**: [16:44] Too much text overlapping, so you could tell that nobody thought about it. So, that was a smell saying that this is not well-done.

**Unsure**: [17:02] Good.

**Unsure**: [17:03] It had a lot of text with a call to action, so that makes it the easiest to verify.

**Anand**: [17:08] Easy to verify. So, ease not just understanding, but verification. Cool.

**Anand**: [17:13] See, what we are now going to do is start contextualizing. You took a chart, no context. Now, we're going to start putting some context behind each of these. And the sheets that I'm going to distribute have the **purpose and the audience** behind individual charts. What I'd like each of you on the table, and whoever's associated yourself with the table if you're not sitting close—just pick some table to associate yourself with—is to read the audience and purpose. Just take a glance at the purpose sheet and the audience sheet.

[18:00] [Background noise of papers being distributed and chairs moving]

**Anand**: [18:16] Take a minute, read it.

[18:18] [Quiet as participants read the context sheets]

**Unsure**: [18:36] You have no idea how much I don't know about charts anymore.

**Anand**: [18:43] That's your chart. Which one? [Laughter]

**Anand**: [18:57] Okay, you probably have a sense, and you can go back to this. Now, each of you will have to make a decision. Each of you being each table. Please discuss over the next five minutes, and **you have to decide whether you want to ship it, fix it, or kill it.** You can say, “Look, for this particular audience, for this particular purpose, this works. Yes, no, whatever.” Or, you could say, “I'm okay to make a change or a few changes,” in which case you have to have a clear sense of **what is the biggest impact change, or highest ROI (less effort, more return) kind of a change that you can make.** Put yourself in the mindset of an editor or a manager whose job it is to say, “Okay, some work, and for this audience and purpose, it’s good to go.” Or, just stop. “I’m not going to send this to my boss. I’m not going to send this to the readers. I’m not going to send this to our clients.”

**Anand**: [20:01] And also, **what is the cheapest, quickest, easiest check that you can make to verify this?** Where applicable. What I'd like you to do is take a blank sheet. As a table, discuss and write two things: **one, whether you would ship, fix, or kill, and why; and how you would verify it.** Decision and why (ship, fix, or kill and why) and verification. All yours. So please just discuss at the table.

[20:38] [The room becomes very noisy as all tables begin talking at once]

**Unsure**: [20:53] Yes, for the general audience.

**Unsure**: [21:03] Shipping should be fine. Why? Because it’s providing enough context and why is it how it is.

**Unsure**: [21:18] This line represents a particular model. Each circle is probably… vertically, each line… No. Actually, each line…

**Anand**: [21:34] Deciding whether you ship, fix, or kill. One of the three. And why.

**Unsure**: [21:46] It’s not a fair… or you don’t have to extract. It’s reinforcing biases or models. Generally stereotypes regarding education. The gap between the urban and rural.

**Unsure**: [22:24] If I ship, I might fix the color. Fix is possible; why would we want to kill or fix?

**Anand**: [22:41] **If you believe that for this particular audience, sharing this could actually cause racial discrimination harm, like on that table, you kill it.** AI could go on, and there are some cases where you say, “Okay, that was an easy one.” Or, this chart itself should not exist. Or, this is the absolute wrong chart for this audience. I would rather recreate from scratch. **Recreation is a kill.** Correct. Etc.

**Anand**: [23:12] But another thing to think about is: supposing a person took a quick glance at it, like you did, **what incorrect hurried conclusion might they draw?** This is something to think about.

**Unsure**: [23:29] One fix or multiple fixes?

**Anand**: [23:31] Two max, please. And also factor in what hurried conclusion a busy reader might incorrectly draw.

**Unsure**: [23:41] And what about the learning curve? You mentioned there’s a learning curve. What should that be? One hour, one week, one month?

**Anand**: [23:47] Fair point. That is part of what I would love to learn from you as part of the discussion. That’s an excellent question.

**Unsure**: [23:55] No, no way! One year learning curve?

[24:06] [Background discussions intensify; snippets of conversations are audible]

**Unsure**: [24:08] I think one thing that was mentioned is they have just two minutes to make this decision. We can’t sit and jot out like explain to them, “Oh look, like this is what’s happening in this way,” etc., where the data is coming from. One thing is, this is a sample size of 15,000 people. There’s a geographic distribution. So, an education department is coming into State Education and they may or may not be looking at this…

**Unsure**: [24:34] Visually, I would like to change something. Visually, it makes absolute no sense. This is data on beauty. Data of beauty is very chaotic; you can’t say… [inaudible] I would prefer my… [inaudible] to simplify for these 15,000 people because they are the main… that study.

---

[00:00] [Intense background discussions as tables decide whether to ship, fix, or kill their assigned charts.]

**Anand**: [11:11] Okay, so now the task on the new table is what's on the screen. Please pay attention to what you need to do. **Critique their decision, but in a nice way.** And what I mean by a nice way is: **find the strongest reason why their decision might be wrong.** Which means you have to understand their audience, their chart, and their sheet and their handwriting. Find the strongest reason why the ship, fix, or kill decision might be wrong.

**Anand**: [11:54] Take another sheet. On that new sheet, write: "**The strongest part of their decision is X. You would change it, if at all, for this reason.**" You don't have to change their decision. You can say, "Fantastic work, no changes." But if you feel like their decision should have been different, write that down.

**Unsure**: [12:24] I have to leave for the tech check, but this was very intriguing so far. Sorry I couldn't attend the rest.

**Anand**: [12:30] No problem.

[12:32] [Tables rotate and discuss the previous group's work. Background chatter remains loud as participants analyze the decisions made by the first group.]

**Unsure**: [13:08] The pricing is correct. They're choosing to fix the... randomly to host the data points...

**Unsure**: [15:03] This is a good model is what you can go and vote based on. It's like schooling. No, IT jobs will have one thing and more skill-based things.

**Unsure**: [15:29] These two are there, financial impact and average income, but these three would be missing. And from this, you can probably infer how important that sector is because that's the amount of salaries they're paying. I'm estimating the cost based on what is the individual cost of a person's starting model.

**Unsure**: [16:16] Cost versus quality, and what if we want to fix it?

**Unsure**: [17:07] A chart of total population... overlaid with a trend line or something, that will still tell me much more than what this table tells me for the question we have.

**Unsure**: [17:28] I wonder if they've interchanged X and Y. Because Chinese is Y-axis, African is X-axis. Here it's the other way. No, don't kill it! It's a good one. No, this graph is fine. The graph is fine as long as you fix the labels. Actually, it shows how absurd the model is, so the model is wrong, but the chart is fine because it reveals it.

**Unsure**: [18:20] Putting the axis in the middle would revealed, yes, how less Asian or less African... If somebody who's done their secondary education then maybe there's a 12% chance that they already have a job. But not a job, a salaried job. Yeah. So maybe the person who is like... what is the chance of getting a job? You are more likely to not have a salaried job...

**Anand**: [19:21] You can write your critiques on the same piece of paper on the back side, or another sheet. Anything's fine.

**Anand**: [20:54] Multiple people have asked that [verification], so we'll cover that in a bit.

**Anand**: [21:04] Okay, looks like a few of you are done. Can just wrap up. Time to finish writing. Write down your critique. Just write down and stay at your tables; we'll discuss. Current table, no need to move. Whichever groups have finished, please just sit down comfortably.

**Anand**: [22:33] All of you can sit down. Take a seat. Now we'll be discussing, so it's best if you take a seat. Go back to your chairs, take a seat. We'll be discussing collectively.

**Unsure**: [23:14] It took us 10 minutes to understand the sheet.

**Anand**: [23:22] That's good. Take a seat. Yeah, you can sit wherever; it doesn't matter. Just let's take a seat.

**Anand**: [23:36] Okay, let's start with Table A and the people who were at Table A. What we're going to do now is discuss amongst ourselves what we found about the other table's work. **What you're seeing now is basically anyone can be giving you visualizations for you to understand, review, and this whole process is the new reality.** Creation is not the bottleneck.

**Anand**: [24:08] AI can create, and random people will start submitting. For instance, I submitted a pull request to a repository for some tool. The way the pull request was created was me asking ChatGPT, "Can you please create a pull request for this issue?" and I submitted it. Within a few minutes, three other bots submitted a pull request for the same issue that I had addressed, and they were trying to get their pull requests accepted.

**Anand**: [24:48] We're seeing that **the ability to create is now so cheap and the quality of creation is not bad, that any Tom, Dick, and Harry will start creating. Which means that the job that remains now—our job has become verification.**

---

This is part 3/3 of the recording.

**Anand**: [00:00] Creation is not the bottleneck anymore. Creation is extremely cheap and anyone can create a lot of things. Reviewing competitors and your own review components—**these are the real tasks for an online platform for the next year. That's what you've got a glimpse of. You're also creating charts; it's not something new.**

**Anand**: [00:32] Now let's discuss collectively and see if we can learn from this part. What we'll do is go group by group. The group that critiqued Table A, step by step, will share their feedback to you, and similarly the next group, etc. Here's how you'll do it: just read out what you have written with minimal additions on top of it. Say, "This was the original decision that they had, this is your revised decision," and then share what you are taking away from it. Not an "I agree/I disagree" point, but rather: **would you change your mind? If yes, why? If not, why not? And what evidence would help you change your mind?** Go ahead.

[01:25] [Participants move to the center and prepare to present their findings.]

**Unsure**: [01:53] Okay, maybe explain to everyone first.

**Unsure**: [01:59] So, the chart was showing salaried jobs for primary, secondary, and higher secondary education—like, from school education level. That's on the blue line. And on the red line, it is showing people waiting for work. "Waiting for work" here is defined as people looking for employment.

**Unsure**: [02:18] Their decision was to fix it, with a specific focus on saying that more granularity is needed, especially at the state level. Because different states have different dynamics in their employment. And their other big fix was focused on the aspect that one of the audience questions was on correlation versus causation. So, that was also a focus that this is not causation; this is just a correlation. That was the clarification which was added.

**Unsure**: [02:53] The critique which we had was that in this case, when we have salaried jobs and waiting for work, even though they are correlated, one large chunk which is not displayed in this is: what about non-salaried jobs? What is that trend looking like?

**Unsure**: [03:15] Our hypothesis, which we'd have to see in the data, our hypothesis is that **people who are only having primary education—the reason that they are not having so much unemployment or looking for employment in the family is because they're easily getting employed in daily wages or in small contractual work, which is not being classified in the blue salaried job.**

**Unsure**: [03:41] And the next level of that is, because of the same thing, we are in agreement with them that it's just a correlation and it's not causation. But what we felt is that the correlation is not in the same vein as the policy kind of recommendation which is happening at the top of the chart.

**Anand**: [03:59] Cool. If you could pass the mic to them. Now, what I'd like you as a group to do is let's reflect for a bit and share collectively or individually, however: **what, therefore, are you changing your mind on?** Put another way, based on an input on your decision set, what are you changing your mind on? What are you not? And also, what evidence will help you change your mind on where you still say, "Ha, maybe not"? Let me recap: what are you saying, "Ha, that is good feedback and therefore is an immediate learning," and second, "Ha, that I would agree only if..."

**Unsure**: [04:49] Something I do agree with is that there should be more detail on non-salaried jobs, and even the categorization. Informal is not called a salaried job, but a little more disaggregation would have helped for sure: what kind of jobs they are entering and why is it that the gap is increasing as the education level is increasing? So, that is something we were unable to really come to a conclusion on in this.

**Unsure**: [05:27] Yeah, I mean one factor could be that the formal sector needs to expand. So, it's just... in the informal sector, the scale is huge; people are easily getting absorbed. But the formal sector is in a state where even if the number of people who are kind of oriented... although also the education level is only till higher secondary, it does not go into professional work or undergrad or anything like that. So, the data set is quite small for that, but it does sort of point towards an action that the formal sector needs to be expanded. But then again, that's one thing we'll need to figure out how to have more of.

**Anand**: [06:13] Is there something that you think you probably wouldn't fix unless... any feedback that you'd say, "Ha, no, I probably wouldn't take that feedback unless..." and if it helps to see the sheet of the fixes?

**Unsure**: [06:33] We were asking them to break the jobs by the sector that they absorb, more like the IT sector. And also not just like splitting the sector, but because the workforce... informal jobs won't be counted as salaried jobs, that should also be plotted across the same thing. Because apart from these three, there's a third category of people who are not actively looking for jobs. So, seeing all four, like the ratio of it in maybe like an area chart, would be more informative of the policy decision.

**Anand**: [07:06] The way, as we progress—for those of you who are listening in—you may not have the full context as much as these two groups of the chart, but look for the texture of what is being evaluated and is there something new in what is being evaluated or different from how you are seeing it? That is effectively part of your learning from the dialogue.

**Anand**: [07:34] Let's move on. What I'd like is for some group that changed the other group's decision between ship, fix, and kill. Anyone change their decision? You changed your decision? And you were initially reviewing this one? You were here and then you reviewed this one. Got it. So, could you share your critique of this? And yeah, please keep that chart visible. If you could explain what this chart was and your critique of their decision.

**Unsure**: [08:10] So, I feel like the context for this chart is to basically... the context of this chart is to tell the decision-makers whether this AI model that they created for a picture—which gives these ratings—is fair to use or not, or to deploy in the real world.

**Unsure**: [08:31] The group had decided to kill the chart. And we decided that the chart is fine because based on the chart's absurdity, the decision-makers can see that since the outliers and the people are not aligned—or like, you know, people are not plotted accurately—the decision could be to kill the AI model, the underlying AI model, and not deploy it in the real world.

**Unsure**: [08:58] However, I think like personally, I felt like there were a few constraints that this chart is working with. One of them being like: why is only Chinese and African the two variables? Like, what about other ethnicities and all? But I feel like it discusses them internally, so maybe that's the constraint that they're working with. But if that is not a constraint, then I personally feel like it's a kill because what about other ethnicities? But yes, we decided to fix the chart based on their decision of killing the chart.

**Anand**: [09:31] **It's interesting that given the same chart, given the same purpose, and given the same audience briefs, we have groups deciding to change the decision in itself, which obviously reflects that even this is not enough.** So, now, we maybe keep this down.

**Anand**: [09:54] Let's talk about that. Based on what this group is sharing—and please feel free to share additional inputs—what would... what are you changing your mind on? And what would you say, "No, I wouldn't yet change my mind on unless I get some other input"? And you feel free to discuss aloud internally if you want.

[10:18] [The group huddles to discuss as background noise from the room picks up.]

**Anand**: [10:23] While that is happening, I'll request each group to just take the sheet that has the critique on your group's work. Just make sure you know where it is or have taken a look at it. Just one person from each group can share. Make sure you have the critique of your review.

[10:55] [Loud shuffling and group chatter as participants retrieve their critique sheets.]

**Anand**: [11:13] 10:45 is when we are supposed to end this session. Is it? The workshop, I thought it was for one and a half hours. And we started at 10:15, which was late, but is it for one hour fifteen minutes or one hour? This is a one-hour session? Are you sure? Because I remember distinctly that I had written down the wrong program... 10:00 to 10:45? 10:00 to 10:45. Okay, share your thoughts please. What would you... No, no, just take the mic in front of you.

**Anand**: [11:53] Sorry, I realized we are way over time. What I had assumed was a one-and-a-half-hour session is apparently a 45-minute session. So, we're like wrapping up like crazy.

**Unsure**: [12:04] So, in order to fix, like, there are a few questions that we'd want to ask here, like: what is the reference group of people that are there and how do you define that? What does the axis imply? It actually plots it on a percentage basis, so how do you define somebody's 30% Chinese and 70% African? That's the underlying AI model; that has nothing to do with the chart.

**Unsure**: [12:26] But even the classification, right? Like, in our audience context, like, if you think like the first two questions which they will ask are... so the first two questions which we chose to fix is that: what do the axes mean? And we thought that given the context of the audience, which is more of a product risk committee or a fairness committee which is from a management background, taking such a complex operationalization of ethnicity into a percentage of someone which a deep learning might understand, it may not answer these two questions: what do the axes really measure and how were the reference groups chosen? That's why we felt that this chart cannot be shipped.

**Unsure**: [13:06] And the other point which we shared is that it's not if we are changing so many things to try to explain it, then why does the chart exist? Honestly, I would kill the whole operation. This segregation never works.

**Anand**: [13:20] And **that division is in my mind showing a couple of things. One, that what we think may not be what someone else thinks even given the same information. But that is the power of the subjectivity and therefore is the thing that AI will not be able to take.** Because if the human training data is in disagreement on this, what constitutes "right" becomes very different.

**Anand**: [13:51] Let me share which of these were AI-generated, which of these were not AI-generated, etc. This one, the first one, **Statoistics**, is 100% AI. Zero human intervention. I was seeing it for the first time yesterday. The second one, **Popular Actors**, is 100% human-generated. I created it—not manually by placing it, but by using embeddings, but I wrote every line of code for this one.

**Anand**: [14:24] This **Temporal Rosette** is 100% AI-generated. It came out as a result of a prompt: "Create a visualization that nobody has ever seen, in fact create half a dozen," and that is the outcome. I have not seen this, not even yesterday, not even today. I have no idea what it's saying.

**Anand**: [14:43] The fourth one is about 50% AI-generated. I said, "This is broadly the theme and I want you to create a chart of some kind," but the output of what it created, I did not review. I said, "Okay, this kind of looks fine, I'm going to publish."

**Anand**: [15:00] The **LLM model pricing** is 100% human. I wrote the code for this, I wrote every single bit of this particular chart. And this is about 50%... this **GDP** one is 50% human-generated, 50% AI. I said, "Create a tree map from this data, but you scrape the data, you put in the stuff," etc.

**Anand**: [15:26] Now, that is the origin, but in all of these cases, they are drafts. What I mean by drafts is **none of them went through a strong editorial process, and that is exactly what you are learning, bringing in, discussing.**

**Unsure**: [15:45] Is it published?

**Anand**: [15:47] It is published. And for those of you who may be from Times of India, or Rohit's probably talking about this right now, this entire property was at the stage where Rohit was saying, "Anand, we probably will have to shut it down because we don't have enough people." So I said, "ChatGPT, create a whole bunch of these Statoistics codex, now take these ideas and implement them."

**Anand**: [16:08] And now the situation is: oh, okay, we don't really have enough people to go through review, validate, etc. We made that process simple. Okay, now we don't have enough space to print. **So the problem completely shifted and it's still shifting. But yes, this I think was published with the team editing the graphics—there are some clear overlap issues, etc., which I didn't bother; I said you can do that.**

**Anand**: [16:34] So, which means that **one of the key things that you will be left with is the ownership. What are people going to take away from this? Are they going to get it wrong? Is it ethically wrong, factually wrong, etc., is something that it's your responsibility to verify. AI may still get it right, but it's your name on it.**

**Anand**: [16:53] Deciding, therefore, what is the kind of evidence that you want... so the Times of India team effectively said, "Look Anand, make this easy for us to verify by telling us how to go about verifying it." So Codex created a list saying, "Go to this particular Excel sheet, go to this particular column, filter by this particular value, you will see that the sum of this extra column is going to be exactly equal to my value." And when they follow that process, it works.

**Anand**: [17:18] So, it's not that the process of identifying what or how to verify needs to be human, but the ownership—somebody saying, Saurabh saying, "I've verified it, I'm okay with this." Whether he's done it or not according to that process, it is that ownership that is going to stay.

**Anand**: [17:38] And after that, when somebody points out saying, "No, look, here is another perspective," or "Here is a factual error," or "No, no, this is not what the audience wants," etc., the person standing up and saying, "Look, I'm accountable. I will take this feedback, I may act on it, I may not act on it," is an ownership that you will have.

**Anand**: [17:59] Keeping that in mind, I'm going to ask you to do one last exercise and after that, you will leave. But before doing that exercise, let me just tell you what you should be taking away from this. **That ownership will last in the AI era. Accountability will last in the AI era. Reviews, validations, etc., will last at least for a few years in this AI era. You might as well practice it.** And practicing is very easy; all you have to do is delegate everything to AI and what is left is what you have to learn.

**Anand**: [18:31] You got a glimpse of this, do more of it. You got a sense of what's working, what's not working from the dialogues. Those takeaways are in fact yours. Now let's exercise that accountability.

**Anand**: [18:43] What I'd like you to do is take your name post-it and, if you choose to, move it or let it stay. But mention one other thing: **are you willing to stand as taking ownership, saying, "I've reviewed this, I'm okay with this"?** Or, "I've reviewed it and I am not okay with it," or "Yes, I'm reviewing it, I'm okay if a change is made," which is basically the ship, fix, or kill.

**Anand**: [19:15] You are allowed to change your post-it. You may say, "Oh, the one that I reviewed, actually I'm willing to commit to, this one I'm not willing to commit to." That is also okay. The net result will be the equivalent of you taking an ownership on somebody else's chart—some human, some AI. And with that, we conclude this session. Thank you.

[19:35] [Applause from the audience.]

**Anand**: [19:37] Go ahead please, just change your name, write and... Oh sorry, we also need a... yeah, skip the group.

[19:51] [Participants stand and begin moving their post-it notes as the session ends.]