What an AI readiness assessment actually looks for

An AI readiness assessment sounds like a sales step we invented to pad a proposal, and we understand the skepticism, because a lot of agencies do use the phrase that way. Ours is a genuinely short, scoped piece of work, usually two to three days, that answers one question directly: is this specific problem worth solving with a model right now, given the data and team you actually have, not the data and team you'd have in an ideal world.

We start by asking to see the actual data the system would run on, not a description of it. A client will often describe their support tickets as well-organized and consistently tagged, and then the export shows three different tagging conventions used across two years by different team members, half the historical resolution notes missing entirely, and no reliable way to tell which tickets were actually resolved correctly versus just closed. None of that makes a project impossible, but it changes the scope significantly, usually adding a data-cleanup phase nobody had budgeted for. We'd rather surface that in a three-day assessment than three weeks into a build.

We also check whether the data reflects the actual distribution of cases the system will see in production, not just the easy, well-documented examples someone pulled together for a demo. A dataset of fifty perfectly labeled examples tells us almost nothing about how a system will handle the messy, ambiguous two percent of cases that show up every week in the real workflow.

The second thing we assess is less technical and more organizational: who on the client's team will actually own this system after we're gone. A workflow with no clear owner tends to degrade quietly, since nobody notices when accuracy drifts, nobody updates the system when a policy changes, and nobody's job depends on the thing continuing to work well. We ask directly, in the assessment itself, whose role this becomes, and if the honest answer is nobody yet, we say so before recommending a build, because that's an organizational gap worth closing first, not a detail to figure out after launch.

We've walked away from otherwise well-scoped projects because this question had no real answer. A well-built system with no owner degrades exactly the same way a poorly-built one does, just more slowly, and we'd rather flag that risk upfront than have a client discover it a year later when nobody can explain why the numbers have drifted.

Every AI readiness assessment includes an explicit conversation about what a wrong answer costs in this specific workflow, because that number should drive the entire design of the system, not just a footnote in the technical spec. A wrong product recommendation costs almost nothing; a wrong answer in a claims or compliance context can cost real money or create real liability. We size the human-review requirements, the fallback behavior, and the confidence thresholds directly against that cost, and it's usually the single factor that determines whether a workflow needs a human in the loop on every decision or just periodic spot-checking.

Clients occasionally push to skip this step because they're confident the failure cases will be rare. Rare isn't the same as tolerable, and a system that's wrong one time in five hundred can still cause serious damage if that one case happens to be the wrong one. We'd rather have this conversation explicitly than let the design default to whatever's fastest to build.

At the end of the assessment, the client gets a short written recommendation: whether the workflow is a good candidate for a model at all, what data work needs to happen first if any, who should own the system once live, and a rough sizing of the actual build. Roughly a third of the assessments we run end with a recommendation not to build an AI system yet, usually because the data isn't ready or the ownership question doesn't have an answer, and we tell clients that is exactly as useful an outcome as a green light. A short, honest no before any code gets written is a much better outcome for everyone than an expensive yes that quietly fails eight months later.

Roughly half of the assessment's second day is spent talking directly to the people who would actually use or be affected by the system, not just the executive sponsoring the project. We've had cases where a proposed support copilot looked straightforward from the manager's description, and then a twenty-minute conversation with three actual support agents revealed that the ticketing system's category field, the one the whole triage logic was supposed to depend on, was routinely miscategorized by agents who'd learned to ignore it years ago because a previous tool had made it meaningless. That's not a detail an org chart or a process document surfaces. It only comes out from talking to the people doing the actual work.

These conversations also tell us how much change management a rollout will actually need, separate from the technical build. A team that's already frustrated with a previous failed automation attempt needs a different rollout plan, more transparency about what the system can and can't do, a longer trial period, more visible human oversight, than a team that's simply never used anything like it before and is cautiously curious. We fold that read directly into the final recommendation, because a technically sound system introduced badly to a skeptical team fails just as surely as a poorly built one, just for organizational reasons instead of technical ones.

We keep a short, deliberately blunt list of questions we ask in every one of these interviews: what do you currently do that you wish you didn't have to, what's the thing you're most afraid an automated system would get wrong, and what would make you actually trust a tool like this. The second question in particular surfaces concerns clients rarely raise unprompted, since staff are often reluctant to voice worries about a project their manager is visibly excited about. Surfacing that concern in a private interview, rather than letting it simmer unaddressed until after launch, has changed the design of more than one system we've built, usually by adding an extra review step exactly where the team told us they were most nervous.