Teams usually design agents by describing what they should do. We start from the opposite end: the complete list of actions the agent is allowed to take, and what happens at the edge of that list.
The tool surface is the contract
An agent can only do what its tools permit. That makes the tool definitions the real security boundary — far more reliable than instructions in a prompt, which are advisory at best.
- Read tools and write tools are separated and separately approved
- Every write tool is idempotent and carries a request identifier
- Destructive actions require an explicit human confirmation step
- Scope is enforced in the tool implementation, never assumed from the prompt
A prompt is a suggestion. A tool signature is a rule.
Escalation is a feature, not a fallback
The best agents we have shipped escalate often in their first weeks. That is the system working: it defers where it is uncertain, a human resolves it, and the resolution becomes a labelled case.
Autonomy then increases on evidence. We raise the threshold only when the evaluation suite shows the agent handling that class of case reliably.
Everything is auditable
For every task an agent completes, an operator should be able to answer three questions: what did it do, what did it read to decide that, and what would have stopped it. If any answer requires reading application logs, the design is not finished.
const bookAppointment = defineTool({
name: "book_appointment",
// Scope is enforced here, not in the system prompt.
scope: ["appointments:write"],
requiresApproval: true,
input: z.object({ patientId: z.string().uuid(), slotId: z.string().uuid() }),
handler: async (input, ctx) => {
await assertTenant(ctx, input.patientId);
return appointments.reserve(input, { idempotencyKey: ctx.taskId });
},
});Design the boundary first and the behaviour becomes a tuning problem. Design the behaviour first and the boundary becomes an incident report.