Delivered, no result
Why a Working System Still Hasn’t Paid Off, and Where the Savings Went.
The system went live after a convincing demonstration and a successful pilot. People are using it, but orders still take too long and managers are still chasing approvals. What happened to the savings?
Before funding more training, customization, or automation, ask: what had to change inside the business for this system to produce the promised result?
By Jim Wilde, Founder of Emergent Skills
Follow the work beyond the demonstration
Start with the result the system was meant to produce, when it was expected, and what has actually changed. Confirm that the result is still worth pursuing and whether a changed priority, missing resource, or outside dependency needs attention first.
Consider a distributor that buys a system to prepare customer quotes faster. This is an illustration, not a client case.
The system does that job. Salespeople can now prepare a draft in minutes. Usage is high, and the implementation is marked complete. Customers are still waiting just as long for final quotes.
Sales still checks every recommendation. Finance still reviews the same discounts. Operations now spends more time correcting delivery promises, because the system doesn't know every operating constraint.
The drafting step improved. Customers aren't getting their quotes any sooner.
The original business case may have underestimated the remaining work. Information may be missing, or decisions may wait for someone who has no time to make them. The new system may also have added work elsewhere that offsets the benefit. These are explanations to check.
Check whether the work needed to produce faster quotes was identified and assigned: maintaining delivery information, agreeing on discount authority, and deciding which old checks to retain. Establish a named owner and the scope of each assignment. An unclear answer needs follow-up. It doesn't prove nobody owns the work.
Find out why the old reviews survived
In Why Even Your Best Employees Override AI Systems That Work, Das Narayandas argues that companies deploy AI systems faster than they redesign the governance around them, so nobody settles who has authority to act on the system's decisions and who answers when one goes wrong.
Think about the employee expected to accept an automated recommendation. If that recommendation goes wrong, will management stand behind the approved process? Or will the employee be asked why they didn't check it?
Keeping the old review may be a sensible response. Before labeling the behavior resistance, ask:
What is this extra check protecting against, and who would have to answer if it were skipped and something went wrong?
Then examine the answer. The check might catch real errors or satisfy a required control. It might cover for missing information. Or it survives because nobody ever authorized removing it.
Each calls for a different decision. Don't remove a review until you know what it protects and how that protection will be maintained if it's still needed.
Count the work before judging the system's payoff
Paul Kedrosky and Azeem Azhar take different views of the AI investment boom in their debate on The Ezra Klein Show. For a business evaluating its own purchase, there's a narrower question: how does a useful tool become a financial payoff?
A tool can save a salesperson an hour while adding checking and correction work for two other people. It can improve accuracy while leaving the main delay untouched.
Time saved may let the team handle more customers. That has value, but if the business case promised lower payroll costs, it hasn't delivered that saving.
Follow the work through to the customer. Count exceptions, corrections, supervision, and any old process still running alongside the new one. Keep extra labor, elapsed waiting time, and the cost of the proposed change separate. If time is released, establish what the business can do with it before assigning a financial benefit.
Give the explanation a chance to be wrong
Suppose the dashboard shows that quotes spend three days in Finance. It's reasonable to ask whether Finance is holding things up. But if Finance is waiting for corrected delivery information, changing its approval process may accomplish very little.
I'd want to look at a few actual quotes before accepting the explanation, especially if everyone has been repeating it since launch.
Ask what evidence would separate the leading explanations.
Compare similar quotes that do and don't need approval. Check whether complete submissions move faster than incomplete ones. If the explanation is that people retain checks because they're still accountable, examine their instructions, escalation decisions, and performance measures. Ask the people doing the work how those requirements affect their decisions.
The comparisons need care. Similar-looking cases can differ in important ways. But they give you something to examine beyond agreement with a plausible story.
Decide what the evidence supports next
The investigation should end with a next step the evidence supports, even if the cause remains uncertain.
That might be a correction: a data source fixed, authority clarified, a duplicate review retired. It might be a limited test, one specific piece of evidence, or a decision to narrow the initiative or stop investing.
Agree on a named owner, the result or learning expected, and a review point. If a test is useful, its length should fit the work and the number of observations needed.
For the distributor, if the evidence supports changing a review, agree on the authority, error limits, and measures before making the change. Another system purchase needs its own case.
Before the next commitment
These questions belong on both sides of a purchase.
Before commitment, ask whether the business case includes the work needed to produce the benefit, and whether the people doing it have the authority, information, and resources. After implementation, investigate what happened and why the expected result is still out of reach.
At Emergent Skills, the Stalled Priority Snapshot investigates one important result that's falling short. It examines necessary work, the path that work takes, and conditions affecting execution, testing competing explanations against the evidence. Before a commitment, Decision Challenge checks whether the evidence supports going ahead and whether the required work is feasible.
Before spending more, take a few recent quotes and follow them from the customer's request to the final response. Find out where they're waiting, what gets corrected, and why people still do the old checks. That gives you a basis for deciding what to change.
Bring one system that isn't paying off.
The Stalled Priority Snapshot is $1,500 fixed for one important result: a 90-minute session with up to three people, followed by a written readout within 24 hours of the session. You receive supported findings and costs, what remains uncertain, and a recommended next step.
