Building an internal copilot that support teams actually use

We've shipped internal copilots for support teams at six different companies over the past two years, and the pattern that determines whether one actually gets used has surprisingly little to do with how accurate the underlying model is. The copilots that get adopted and the ones that get quietly ignored after week two are usually built on comparably capable models. What differs is how the tool fits into an agent's actual workflow, not the model behind it.

The single biggest adoption factor we've found is whether the copilot lives inside the tool support agents already use all day, their helpdesk software, or requires them to open a separate tab, log in somewhere else, and copy information back and forth. A copilot that requires a context switch gets used for the hardest five percent of tickets, when someone is stuck enough to bother, and ignored for the other ninety-five percent, even when it would have saved real time on those too. We build directly inside the existing helpdesk now, as a sidebar or an inline suggestion, specifically because that removes the activation energy that kills adoption before the tool ever gets a fair evaluation from the team using it.

Every support copilot we've built starts in suggest mode: it drafts a response, an agent reviews and edits it, and only the agent sends it. We've had clients ask to skip straight to auto-send for routine tickets to save more time, and we push back on that for at least the first two months of any rollout, because auto-send removes the feedback loop that tells us where the model is actually weak. An agent who edits a draft before sending is unconsciously giving us labeled training data on every single ticket, showing exactly where the draft was close and where it missed. Move to auto-send too early and that signal disappears, along with the trust of any agent who gets blindsided by a response they never reviewed going out under their name.

Every team where adoption happened fastest ran a short session before rollout where senior agents reviewed a batch of the copilot's draft responses together and openly discussed what was good, what was off, and why. That session did two things: it gave the team a shared, calibrated sense of what the tool was actually good at, rather than each agent forming an opinion from a handful of individual interactions, and it surfaced specific, fixable failure patterns we could address before the wider team ever saw them. Skipping that session and just rolling the tool out to everyone at once consistently produced slower, patchier adoption, driven by a few bad early interactions that colored an agent's opinion of the whole tool before it had a fair chance.

Accuracy metrics on the model's output matter less to us at this point than two adoption metrics: the percentage of drafts agents actually use with minor or no edits, and the percentage of tickets where an agent opens the copilot at all when it's available. A model can score well on an internal accuracy benchmark and still fail if agents aren't opening it, and that second number is the one that tells us whether the tool is actually solving a real problem in an agent's actual day rather than a theoretical one we assumed on their behalf. We report both numbers to clients monthly, alongside the more traditional time-saved metric, because time-saved is meaningless if the tool isn't actually being used to get there.

The scenario every support lead worries about before launch is the copilot drafting something wrong that an agent sends without catching, especially in a support context where wrong can mean giving a customer incorrect information about a policy, a refund, or a service limitation. We address this less through model accuracy and more through interface design: the draft is visually distinguished from an agent's own typed text, sits in a clearly labeled suggestion panel rather than pre-filling the reply box directly, and requires an explicit action to accept and edit it rather than a single click to send as-is. That extra half-second of friction is deliberate, and it consistently correlates with lower error rates than a more frictionless design that pastes the suggestion straight into the reply box ready to send.

We ran one rollout where a well-meaning team lead disabled the review-required setting after two weeks because agents had stopped editing drafts noticeably, taking that as a sign accuracy was high enough for auto-send. We'd recommended against that specific change, since low edit rates can mean the copilot is genuinely accurate, or it can mean agents have started trusting it more than the evidence actually supports and are skimming rather than reviewing. Within three weeks, a factually wrong policy statement went out to a customer, was corrected within the hour once flagged, but reinforced our position that review-required should stay in place for considerably longer than adoption metrics alone would suggest, closer to a full quarter of stable performance than the two weeks the team lead had judged as sufficient.

Since that incident, every rollout we run keeps the review-required setting in place for a minimum ninety days regardless of how quickly edit rates drop, treating time and volume as a floor rather than letting early good performance shorten it. We tell clients this upfront now, before rollout even begins, specifically so a fast drop in edit rates reads as encouraging progress rather than a signal to relax the review step early, since the two are easy to conflate in the moment and only one of them is actually safe to act on. It's a short conversation to have before launch, and it has consistently saved us a much harder one after something has already gone out to a real customer.