The Problem We Are Creating
We are building and accepting a world full of unidentified agents with autonomous decision-making power and without an explicit Human-Agent Relationship.
Let’s imagine that you ask your agent, “Book me a flight ticket to Stockholm tomorrow, arriving before 12:00PM under 300EUR”, and you believe you’ve clearly expressed your intent. The agent will decide which airline to book, whether to include luggage, which seat to choose, whether to buy an insurance policy, and which payment method to use. Your initial intent has silently delegated decisions to the agent that you never explicitly made. Those are autonomous decisions.
To complete the purchase, the agent acts on your behalf. It authenticates as you, accesses your accounts, and uses your identity to interact with external services. The service never sees the identity of the agent making decisions on your behalf. This is an unidentified agent.
More importantly, the service never sees the relationship between you and that agent. It cannot know whether the agent is authorized to act for you, what decisions it may make, or whether that authority is still valid. That is a Human-Agent Relationship.
Unfortunately, this is how we’re designing AI agents and the tools around them: to be unidentified, to have autonomous decision-making power, and to not have a Human-Agent Relationship.
We just ask agents to do this or that. In the end, an agent is making decisions and acting only because a human has chosen to let it act without constraints. Simple as that. Don’t get me wrong. In some edge cases, letting agents make decisions, or better, letting them help you take decisions, could be helpful, but the action that derives from it should at least be approved by the human. But if we, as humans, do not want to let these agents act on our behalf and act anonymously, why are we accepting a world of anonymous agents by also letting them have autonomous decision-making?
The Missing Primitive: Human-Agent Relationship
Today’s focus is mainly on better models, better harnesses, better memory, more context, and tools. These are very important improvements, but they are not the missing piece. What we’re missing is a way to explicitly model the relationship between a human and an agent. That’s why I think it’s time to think and focus on a new fundamental: the Human-Agent Relationship.
I ended up defining the Human-Agent Relationship as: The trusted relationship that defines how a human and an agent are connected, what they represent to each other, and how that relationship evolves throughout its lifecycle
Here are some questions that I keep coming back to, and I think they will help define this Human-Agent Relationship better.
- Who does this agent represent?
- Who delegated its authority?
- What is it authorized to decide?
- Can that authority be verified?
- Can that authority be revoked?
- Who is ultimately accountable for its actions?
- Can another party trust that relationship?
- Can an agent represent more than one human?
- Can a human delegate to more than one agent?
- Can authority be transferred or inherited?
- Can two humans safely share one agent?
- When does the relationship begin? When does it end?
I haven’t found good answers to all these questions because we lack a common language. Before trying to answer them, I first needed to define that language.
The Chain of Legitimacy
Before going further, I would like to define a few terms so we have a common vocabulary, also useful for future articles. These definitions apply equally to humans and agents:
- Identity: uniquely identifies an actor
- Delegation: the relationship through which an actor represents another
- Authority: the permissions granted to an actor within that relationship
I like to refer to them as a Chain of Legitimacy because the real concern is not the execution itself here, but rather whether the action is legitimate.
The Chain of Legitimacy provides three fundamental properties:
- Explainability, letting us explain an action by tracing it backwards
- Accountability, letting us trace back to the delegating and delegated identity
- Trust, letting us know that every requested action is legitimate
We may spend the next decade building relationships between humans and agents. Because in the agentic world, trust does not belong only to agents. It belongs to Human-Agent Relationships.