5 Signs Your Agentforce POC Will Fail — and How to Measure Success Before You Build
Most Agentforce proofs-of-concept don’t fail because the technology can’t do the job. They fail for reasons you can spot before a single line of code is written — and the biggest one is that nobody defined how success would be measured. Here are the five warning signs, plus a practical framework for measuring an agent the right way.
The five signs your POC is heading for the graveyard
1. Dirty data
You’re grounding the agent on a CRM nobody has cleaned. Duplicate accounts, blank required fields, opportunity stages that haven’t been true in months. The model doesn’t perceive this as “messy” — it treats your records as ground truth and answers confidently wrong. Fix the data the agent will touch before you build. Grounding quality is the ceiling on output quality.
2. No measurable goal
“Improve customer service” is a wish, not a target. If you can’t attach a number to the job — deflection rate, response time, hours saved — you’ll never be able to prove the agent worked, and pilots that can’t prove value quietly lose their budget. We’ll come back to exactly what to measure below.
3. Over-broad permissions
Giving the agent write access “just in case” is how a single agent modifies thousands of records before anyone notices. Treat agent permissions like a new integration user: least privilege, scoped to the objects and actions the job needs, with guardrails on writes.
4. Boiling the ocean
Trying to automate everything at once is the fastest way to ship nothing. Scope creep kills POCs faster than any technical limitation. Pick one narrow, valuable job and do it completely.
5. No baseline
If you never measured “before,” you can’t demonstrate “after.” No baseline means no business case, which means no funding for phase two. This is the most avoidable failure on the list — and it leads directly into how you should be measuring.
How to measure an agent the right way
Measurement isn’t something you bolt on at the end. It’s how you scope the POC in the first place. Here’s the approach we use with clients.
Step 1: Pick one primary metric that maps to the job
Every agent should have exactly one number that defines success. Not five — one. That metric should map directly to the business outcome the agent exists to improve. If you’re deflecting tier-1 tickets, the primary metric is the percentage of tickets resolved without a human. If you’re speeding up account prep, it’s hours saved per rep per week.
Step 2: Pair it with a guardrail metric
A primary metric alone is dangerous. An agent can “deflect” 90% of tickets by giving confident, wrong answers. That’s why every primary metric needs a guardrail — a quality or safety measure that stops you from optimizing the headline number at the expense of trust. Deflection is guarded by escalation rate and CSAT. Hours saved is guarded by a spot-check on summary accuracy.
Step 3: Capture a baseline before go-live
For roughly two weeks before the agent goes live, measure the same metric under the same conditions with humans doing the work. This is your baseline. Without it, any number the agent produces is meaningless — you’ll have no way to say whether it helped. Keep the sample honest: same team, same time window, same definition of “resolved.”
Step 4: Separate leading from lagging indicators
Lagging indicators (CSAT, revenue influence, cost per resolution) tell you whether it worked, but they move slowly. Leading indicators (containment rate, response time, task completion rate) move within days and tell you early whether you’re on track. Watch the leading indicators during the POC so you can course-correct before the lagging ones confirm success or failure.
Step 5: Set a threshold and a decision date
Decide up front what “good” looks like and when you’ll judge it. For example: “By day 30, the agent handles 40% of tier-1 tickets with an escalation rate under 10% and CSAT no worse than baseline.” A clear threshold turns a vague pilot into a decision you can actually make — expand, iterate, or stop.
Putting it together
The pattern behind every successful agent we’ve shipped is the same: one narrow job, clean grounding data, tight permissions, and a measurement plan defined before the build. Start smaller than feels impressive, prove the number, and expand from a position of evidence.
If your first sign is any of the five above, pause and fix it now — it’s far cheaper than rescuing a POC after it’s stalled. (For more on why agents stall in production, see our piece on why most Agentforce projects stall, and our plain-English guide to how MCP fits into the picture.)
Shrine Software Services builds enterprise AI agents and Salesforce, Agentforce, Zoho, Java, and React solutions for clients across the US, UK, Australia, and India. Scoping a POC and want a measurement plan that survives the budget conversation? Let’s talk.