What made Sammy 3 work

Sammy 3 was a support agent at Sammy Labs. It shared the user's screen, talked with them, and moved through the product alongside them. It started as a prototype to shorten customer onboarding and became a core part of the support workflow. A few things made the difference between a demo and something people relied on.

Context beat cleverness

The agent's quality tracked how much it knew about the moment it was in: which screen the user was on, what they had just done, and what they were trying to finish. With that, ordinary models made good decisions. Without it, better models still asked the user to repeat themselves.

Most of the engineering went into giving the agent that picture. We mapped the product's interface into a graph it could navigate, so "where am I and what can I do from here" had a concrete answer instead of a guess from pixels.

Memory needed layers

Retrieving text chunks by similarity was not enough. Chunks carry no sense of order, intent, or what is happening right now. We split memory into three parts:

• what is happening in this session;
• what happened with this user before;
• what is true about the product in general.

Each part was written and read differently. The win was not any one store; it was the agent drawing on all three at once, which is what let a conversation pick up where the last one ended.

Tracing came first

The hard part of an agent is not getting it to act. It is understanding why it acted the way it did. We built tracing into every step, the inputs the agent saw, the tools it called, and what came back, and the debugging tools around it made iteration about three times faster.

Once that existed, prompt changes stopped being guesses. We could see the failing step, change one thing, and check the same case again.

Smaller prompts

Early prompts tried to anticipate everything and got worse as they grew. Short, constrained instructions with good context in front of them worked better than long instructions with thin context.

Handoff

When the agent passed a user to a human, the context had to go with them. A human who has to ask "what's the problem?" undoes everything the agent did. We carried the session state across, so the person picking it up could continue rather than restart.

The lesson I keep using: measure outcomes like time to activation and resolution, but also watch the sessions. Users could always tell us when the agent felt like it understood them, and that signal led us to the right fixes faster than any metric did.