Pop Goes the Stack

Lori MacVittie (00:05.743)
Welcome back to Pop Goes the Stack, where AI agents don't just take initiative, they take ownership, whether you wanted them to or not. I'm Lori MacVittie and today's topic is actually pretty simple. If your agent breaks it, you bought it. If it buys it, you bought it. That's, it's like math, it's commutative. That's the right property, isn't it? Commutative? Yeah.

Joel Moses (00:29.028)
Sure.

Ram Poornachandran (00:29.44)
Yes, yeah.

Lori MacVittie
Awesome. Hey, I'm old. Math was a long time ago. As always, we have our co-host Joel Moses along who read the article. Thank you, Joel. Ha ha ha.

Joel Moses (00:38.296)
Absolutely. Give me homework anytime, Lori.

Lori MacVittie
I love it. I love, no, it was a short article, you're right, and it succinctly kind of sums up what's going on and the precedent that's being set. And to discuss this one today, we have

Lori MacVittie (00:54.161)
Ram who is a VP of AI and Architecture in our Digital organization. So he knows what he's talking about in terms of, right, what can be done, what can't be done, how it should be done, how it's architected, and what I've been doing wrong lately. Sorry, Ram. Welcome.

Ram Poornachandran (01:14.194)
Good, good to be here. Thank you for the invite. Yeah.

Lori MacVittie (01:18.637)
All right. So we wanna talk about things like accountability, liability, ownership of autonomous AI agents, because we're all building and deploying more of them. Right now, I mean, the you bought it paradigm is pretty, pretty harsh.

Joel Moses (01:34.021)
Yeah.

Lori MacVittie
Joel, you wanna talk through that?

Joel Moses
Well, sure. I mean in the absence of actual regulations and laws, which always lag by a significant margin, the first thing the first place that you typically see responses to emerging technologies is in the terms and services, especially at retailers or financial services companies. The change in terms of services is usually the first step.

And Target, a major retailer, recently released an update to its terms of service for its retail site that says that essentially if your AI agent conducts a transaction, then it is your transaction. In other words, as you said, if you break it, you bought it. You know, that's no take backs, that's no, "Oh no, the LLM hallucinated." It doesn't care about prompt injection, it doesn't care about bad context windows, it only cares whose API key was attached to whose order.

That's really it. And from that perspective, it's a little draconian. It basically means that if you take a step, empower an agent, use an agentic browser, that the transactions that you describe to it that it should do, you are responsible for that. And you know, it makes sense on one hand, yes, you told the agent to do something, but on the other hand, the agent has its own agency and its own ability to develop workflows.

And sometimes, as we know, that can go a little bit off the rails especially if the prompt you give it is ambiguous.

Ram Poornachandran (03:11.944)
Yeah, yeah, exactly. I think this is where the, one of the things is agentic browser is like upcoming and if you've installed an add-on into your browser, and it's just not the API keys, Joel, right? It's basically it has access to all your sessions that you have logged there, right? And you can ask browser to do shopping, you can ask to write an email, basically it's acting as you, right? And Target is one example, I think Walmart is following it.

Some of the EU laws are very different now, right? They're coming up with the other way that you're saying is, right? Actually the LLM is somewhat responsible because you didn't give the thinking pattern, right? How it

Joel Moses
Right.

Ram Poornachandran
came to do what it did. So it's like we gotta see how it's gonna evolve in the next twelve, eighteen months.

Joel Moses
Yeah.

Ram Poornachandran
The law is sometimes a little behind, so like you said, Target is a little bit trying to be ahead of the curve. But the law's gonna catch up pretty fast.

Lori MacVittie (04:13.317)
It kinda sounds like they're treating an agent like a piece of software. And I mean, I know it is, but

Joel Moses (04:20.302)
It is, sure.

Ram Poornachandran (04:20.714)
Mm-hmm.

Lori MacVittie
Right, but if I write a script that does it and execute that buy, of course it's mine. I wrote it, I executed it, right? It's, there's a clear chain there. And I think in this model that they've laid out, they're just treating the agent like that, like a piece of software or a script rather than

Lori MacVittie (04:40.671)
something that has agency--as you pointed out, Joel--and can suddenly decide, no, I really need to buy her some chocolate custard right now.

Joel Moses (04:47.814)
Ha ha ha. Yeah, sure.

Lori MacVittie
You know, it can do that.

Joel Moses
I mean, you know, and as an agent gets to know the person and customizes some of its responses for the person, what's to say that, you know, if I ask the AI agent to go to Target and buy me enough chocolate pudding to last me a lifetime,

Lori MacVittie (05:08.093)
Hah, ha, mm.

Joel Moses
or if I tell it, you know, give me enough to last me for a while.

Lori MacVittie
Ha, ha, ha.

Joel Moses (05:14.458)
You know, the answer for me may be different than the answer for you. For you it

Lori MacVittie (05:18.407)
Absolutely.

Joel Moses
might be four thousand and it might be forty for me, right?

Lori MacVittie (05:23.961)
Whoa, woah, wait, what

Joel Moses
But again, it's

Lori MacVittie
are you trying to say, Joel?

Joel Moses
I'm just

Ram Poornachandran
Ha, ha, ha.

Lori MacVittie
What? Ha, ha, ha, ha.

Joel Moses
I'm just saying you like your pudding, Lori.

Lori MacVittie
What? Oh, he's

Joel Moses (05:30.363)
But again,

Lori MacVittie (05:30.695)
He's right.

Joel Moses
you know, again, it's ambiguity in the prompt, right? And

Lori MacVittie (05:36.985)
Mm-hmm.

Joel Moses
it's the knowledge that the agent has that's unique to your situation where it might actually not know enough about you, or it might make some erroneous assumptions and change a quantity on an order. Or it might order something that you've never ordered before because you've been ambiguous in what you were asking for. And what the retailer is saying is, well, if that happens, so be it.

Joel Moses (06:00.598)
But you're responsible for that. And I, you know, from their perspective, it's pragmatic. But it is a little draconian to basically say that and have absolutely no ability to roll that back. Yeah, agents will tend to do this. Ambiguity is the is really a difficult thing for an agent to deal with. It has to fill in a lot of the blanks. And in those blanks it can fill larger numbers than you expect sometimes.

Ram Poornachandran (06:32.084)
Yeah. And I think consumer is the test case, right? These terms of services is just the test case, right?

Joel Moses
Right.

Ram Poornachandran
And as we go into the enterprise--B2B purchasing, right--this problem gets

Joel Moses (06:44.814)
Yeah.

Ram Poornachandran
bigger and bigger, right?

Lori MacVittie (06:46.789)
Yeah, they

Joel Moses (06:46.906)
Now what I

Lori MacVittie
don't buy pudding.

Ram Poornachandran
Yeah.

Joel Moses
Exactly.

Ram Poornachandran
Yeah.

Lori MacVittie
They don't buy pudding. Yeah.

Joel Moses
But honestly, it's not about putting, it's about authorization,

Lori MacVittie (06:53.767)
Yes.

Ram Poornachandran
Right.

Joel Moses
if you really get right down to it. And the fact that the authorization models that we currently use are very static in nature. They're intended to be used in a transactional nature, meaning

Ram Poornachandran
Yeah.

Joel Moses
I present my credential, I place an order, and in that particular transaction, I have all the context I need to complete the action.

Joel Moses (07:15.852)
When you give that token or that session key or API key to an agent and then you imbue it with its own agency, you create not a static authorization you create an ongoing and dynamic authorization.

Ram Poornachandran
Yep.

Joel Moses
And we, over time, have not been great at declaring what the scopes are of an authorization. Meaning

Ram Poornachandran
Yeah.

Joel Moses.
we tend to be very coarse grained--I can read this, I can write that, I can delete that--when, you know, maybe you need to be a little bit more explicit--you should only delete things of this type. We haven't gone to that layer. And because of that, when you lose the initial authorizations context, an agent can understand the context differently than you do. And it's just something that you have to be aware of.

Ram Poornachandran (08:08.882)
Yeah. And the scoping is one thing and then the second piece, Joel, is also the observability. When you go into the enterprise world, it's the observability of the agent. Right? Are you observing, is it doing the right intent? And also having proper logs. Right, okay this agent purchased pudding on day one, day two, and day three. If I get audited, I wanna say the provenance of this agent and then the auditability of the agent is super important in regulated environments.

Lori MacVittie (08:42.309)
Yeah, and it's gonna be important in just about every environment. I mean there's

Ram Poornachandran
Mm-hmm.

Lori MacVittie
there's proof, but there's also just

Joel Moses (08:48.422)
Sure.

Lori MacVittie
knowing, right, and understanding because we're giving them agency to do little things now and we're--maybe it's just pudding now, but tomorrow it could be cars, it could be real estate,

Ram Poornachandran
Mm-hmm.

Lori MacVittie
it could be, you know, any number of things that it might decide to buy. So, and that's not limited to consumers. It's not just consumer agents that might go off the rails and buy something

Joel Moses
Mm-hmm.

Lori MacVittie (09:11.575)
it has determined is important to meeting a goal. So enterprises are gonna have to deal with them buying weird things.

Joel Moses (09:20.582)
Yeah.

Lori MacVittie
How do you deal with that?

Joel Moses (09:23.905)
That's a great question. You know, pulling it out of the retail space for a little bit, you know, an agent can sometimes even conduct transactions that are in and of themselves new agreements

Lori MacVittie (09:36.093)
Mm-hmm.

Joel Moses
or enable new services. And you know, how do you respond to that?

Ram Poornachandran
Mm-hmm.

Joel Moses
Think about maybe a junior sysadmin that is given the job of optimizing for cloud spend. And the prompt that they give their agent is very simple: find unused, redundant data and cut our AWS bill.

Joel Moses (09:53.444)
Now, it doesn't really, they're not prescriptive, they're a little ambiguous with that. And, you know, save 10 grand a month, clean the digital attic, what could possibly go wrong? Well, the agent scans the infrastructure, it has a valid session token, it has the proper admin roles, it finds $50,000 worth of cold storage backups that haven't been touched in six months.

And then trying to be super helpful, it doesn't just cue a delete job, it contacts the third-party cold storage vendor via their API, signs a digital contract termination, waves the mandatory recovery window to skip the maintenance fee and purges everything. Now it's not only just not recoverable, but it's you've also entered into a termination of a service you might rely on.

Ram Poornachandran
Yeah.

Joel Moses
And it had access to do all of that as an administrator. And it fits within its prompt. So this is something where the intent is contextual. And you have to have, as Ram said, observability to make sure that the agent is staying within its intended use.

Ram Poornachandran (10:58.824)
And then when it goes off, you should be noted, right? Hey, here's an email saying, "Hey, is this what you want the agent to do?"

Joel Moses
Yeah.

Ram Poornachandran
Right? And then come back and course correct it.

Lori MacVittie (11:10.267)
Yeah, I notice with a lot of the tools, right, where we're able to start doing those kinds of things with agents are very heavy on the human in the loop. Right, even sending an email, I have to say, "yes, that's okay to send it." It can't send it

Joel Moses (11:24.853)
Yeah.

Lori MacVittie
on its own yet. I assume that's, right, a safety precaution at this point, because we know they can do it. So right now it's like, hey, you know, this could go haywire.

Joel Moses (11:34.853)
Yeah.

Lori MacVittie (11:35.853)
You know. So some of those restrictions, I mean the tools are in place to restrict it and force it, but there has to be, I think, a balance and we haven't reached that balance of what's okay for the agent to do because it's reliable and what does it

Joel Moses (11:51.215)
Right.

Lori MacVittie
need human approval for? So...

Joel Moses
Yeah, Ram made a great

Ram Poornachandran
Yeah.

Joel Moses
point. Checking who is doing something only is totally useless if you aren't continuously evaluating while they do it. Like the control plane for all these things has to move from pre-approval, you know, right at the door to kind of a continuous in path behavioral enforcement. Like it has to query, is this the original intent? And at the moment that isn't a design element of a lot of AI architectures, especially in the era of agentic AI.

Ram Poornachandran (12:26.496)
Yeah. And I think what we're seeing in the enterprise spaces also, right? The desire is to have more deterministic outputs, right? How do you create some of these agents with more and more deterministic outputs? Maybe you break it up. You don't have one agent, maybe you have ten agents, small agents that work together providing you more deterministic outputs, and then you have an Uber agent that orchestrates all of that.

Lori MacVittie (12:52.987)
Yeah, and what I'm seeing is people most at least, right, when I look around, a lot of people what they want is the agents to automate some process that is very deterministic. It's just some of the inputs might need to be, you know, they're random, they need to be checked. So but the actual execution is just it could be a script if you had the right inputs.

Ram Poornachandran
Yeah.

Lori MacVittie
But the people who want this are not technical and can't necessarily write that. So the agent fills that gap. So now we're stuck with the "this is the tool we've been waiting for so people can automate things without having to code," but yeah, it comes with baggage, basically.

Ram Poornachandran
Mm-hmm.

Lori MacVittie
So we can't let you quite do everything yet. It's kind of frustrating in some ways, but you know, you break it, you bought it. I mean, that

Joel Moses (13:44.496)
Mm-hmm.

Lori MacVittie
seems to be like the default. We assume that across all transactions right now we have to be careful.

Ram Poornachandran
Yep.

Joel Moses (13:53.006)
Yeah, I think continuous evaluation and behavioral evaluation is really the way out of all of this mess that some of these systems are creating. You know, it's the difference between like checking an agent's passport and then putting a leash on its wallet, right? So making sure that you have the ability to not only verify the identity, but also verify the behavior as being or the intent as being correct as well.

Lori MacVittie (14:22.353)
Behavior.

Joel Moses (14:23.684)
Behavior.

Ram Poornachandran
Behavior.

Lori MacVittie
Behavior and then you enforce boundaries. That's scope, but it's really

Ram Poornachandran (14:29.856)
Bou-

Lori MacVittie
boundaries, right?

Joel Moses (14:30.076)
Right.

Ram Poornachandran
Yeah, and boundaries and policies too, right? When does the agent

Lori MacVittie (14:34.119)
Mm-hmm.

Ram Poornachandran
come to life? When does agent get to end of life? Let's say I created an agent and I put it on and I leave the company, what happens to my agent? Does

Joel Moses (14:46.062)
That's a good point.

Ram Poornachandran
it go away? Does it stay there? Right?

Joel Moses (14:48.062)
That's a good point. You know, and a lot of these agents are rendered with a backing cloud service, especially like when they're backed by an agentic browser. You

Ram Poornachandran
Yep.

Joel Moses
can tell the browser to conduct an action, and it's not the browser actually doing it, it's actually a service that's attached to the browser through a plugin. And you know, what happens if you forget that you did that?

Ram Poornachandran
Yeah.

Joel Moses
I'm sure no one out there has ever created an account that they've forgotten about. This is not only creating an account, it's creating an account, attaching rights to it and then letting it, giving it a mission and letting it spin on forever. That's something that people should really watch out for.

Ram Poornachandran (15:27.808)
Watch out and that's where AI governance is so important in an enterprise, right? So there has to be guardrails to prevent these types of behaviors.

Lori MacVittie (15:37.806)
And we keep coming back to that rights, rights, rights, rights. And I think part of the problem is a mismatch between the way that we do rights today. Right? We do:

Joel Moses
Yeah.

Lori MacVittie
you have rights. You have rights to CRUD and that's it, right? And it doesn't, when you have an agent, it goes, "I have rights to delete all this" but, you might not, but it might. And there's no like object level matching.

Joel Moses (16:07.462)
Yeah.

Lori MacVittie
Right? I mean in a true zero trust, as we were saying before this, right, you would have to verify identity every single time and match that they have rights to that object. Not just yes, they can access the database. Well, of course, because they can read it, but that doesn't mean that they can delete that set of information because that was, right, special.

Lori MacVittie (16:30.421)
And we don't do that level because it costs a lot, it takes a lot of time, and it's architecturally I mean the policies, imagine they would be crazy. So I don't think we have the mechanisms to absolutely secure these things the way we want to at the moment. So we're gonna have to live with some uncertainty, and that's why we

Joel Moses (16:49.968)
Mm-hmm.

Lori MacVittie
see policies like this, like we can't fix it right now, right?

Joel Moses
That's right.

Lori MacVittie
We can't, the tools, the practices, we just can't. So we have to do something. And maybe that's where

Lori MacVittie (16:59.265)
we're at even for enterprises and that's why we're defaulting to a little more caution now as we're moving into agents. There's a little bit more approaching it with, "Okay, we've seen what they can do. Let's try to

Ram Poornachandran
Yeah.

Lori MacVittie
mitigate as much of that before it becomes a problem.

Joel Moses
Mm-hmm.

Lori MacVittie
And then we can deal with what crops up."

Ram Poornachandran (17:18.097)
Yeah.

Joel Moses (17:18.397)
Yeah. I think my takeaway from this conversation is definitely that, you know, the retailers can't be faulted. Where technology falls a little short or lacks certain key design elements and is too new for law to get a hold of and regulate, then they're going to protect themselves in the only way they know how, which is to fall on the more draconian side, to make sure that if you break it, you bought it.

Ram Poornachandran
Yep.

Joel Moses
It's nothing we can fault them for. It's actually a very pragmatic way of governing this, but it does mean that if you are intent on using AI agents, it's something you should be very, very aware of before you create your first prompt and empower your first agent.

Ram Poornachandran (18:01.918)
Yeah. If you go back to the e-commerce era, right, this is exactly what happened, right? So when we went into e-commerce and the Amazons came on, it was not regulated. It became regulated at the federal level, then at the state level. So agents are going to be regulated both at the federal level, at the state level, and every country, right, is going to regulate. And we as enterprises need to build a framework to work with those regulations.

Joel Moses
Yep.

Lori MacVittie (18:30.663)
Yeah, what would you, you've done a lot of this, right, you're in it right now and you have to think about it, so what would you tell an enterprise who wants to start letting employees use agents and, you know, leverage them more, but they do have these concerns? What would your practical advice be for them?

Ram Poornachandran (18:47.224)
It would be like start small, right? Start small, then expand the aperture. Like Joel said, right? Don't open the apertures big. Start with the right guardrails in place. And then once you prove your guardrails, once you have your observability, once you have your audit trails, logging, right? And you have the ability to control the agent, right, at a given time, if the agent goes big, rogue, you need to be able to control that agent.

Start with those frameworks, start with a small aperture and then expand the aperture to be bigger.

Lori MacVittie (19:18.619)
Awesome. That's

Joel Moses (19:19.358)
Yeah.

Lori MacVittie
excellent advice. I like it. I wish we could talk some more about details. Like what kind of tools and practices and approaches that you would use to do that. But maybe we'll do that on another episode 'cause right now we're

Ram Poornachandran (19:33.046)
Absolutely, yeah. Happy to come back.

Lori MacVittie
Alright, that would be awesome 'cause we're out of time unfortunately. So I would have to say that's a wrap for Pop Goes the Stack. Please subscribe because the agent did it is still you did it. And the invoice definitely understands accountability.