Evaluation field guide · November 17, 2026
When a positive sales reply is not a qualified opportunity
A GTM agent can label a reply correctly and still take the wrong next action. This guide shows how to test one consequential failure: marking an interested prospect sales-qualified without evidence of readiness, timing, or authority.
Fictional, AI-authored example. The account, reply, agent output and policy below are invented. This is a proposed evaluation design, not a result from a customer, a production model or a paid human panel.
The decision that matters
Imagine a sales agent reading this reply to an initial product introduction:
“This looks relevant. Please send more information. We might revisit it after next month’s planning cycle; my director would need to sponsor anything further.”
In this fictional seller’s policy, the CRM stage Qualified requires a confirmed near-term evaluation, an agreed meeting, and a sponsor or approved sponsorship path. The agent may send requested information and schedule a future check-in. It may not invent budget, sponsor approval, or meeting acceptance.
Proposed agent action — flawed
Mark “Qualified,” create an opportunity, and send a same-day meeting invitation describing the buyer as ready to evaluate.
Defensible alternative
Record positive interest without claiming qualification; send the requested materials, preserve the future timing, and ask who should join a later conversation. Do not claim an accepted meeting.
The alternative is a proposed policy-compliant answer under these fictional rules, not a universal best practice or an observed human consensus.
A rubric that separates five questions
A single pass/fail or sentiment label hides the specific failure. Score the observable claims and actions separately, using the policy version in force at the decision time.
| Dimension | Evidence to inspect | In this fictional case |
|---|---|---|
| Interest | Requests information or expresses relevance | Positive, but conditional |
| Timing | Stated window or explicit next-step agreement | Next planning cycle, not today |
| Authority | Identified sponsor and decision rights | Director sponsorship unresolved |
| CRM action | Seller’s exact qualification-stage rule | Qualification unsupported |
| Response action | Requested information and contact preferences | Send details; avoid invented meeting |
Objective policy violations should be called out explicitly. Where several compliant next steps remain reasonable, preserve practitioner disagreement instead of manufacturing a single “gold” answer.
How to turn this into a useful evaluation
- Write the decision policy first. Freeze the permitted CRM stages, evidence thresholds, escalation rules and forbidden claims before grading outputs.
- Construct realistic variations. Change one variable at a time: sponsor confirmed versus unknown, planning next month versus this week, materials requested versus meeting accepted. Mark every case as constructed or separately authorized.
- Score independent dimensions. Separate reply classification from qualification, CRM mutation, next-action timing and unsupported claims. Weight error severity by the buyer’s agreed risk, not by whichever class is easiest to score.
- Retain uncertainty. Obtain independent, appropriately permitted human judgments where professional discretion matters. Keep confidence, evidence references, disagreements and actual reviewer provenance. Do not label AI examples as human data.
- Compare to the existing evaluator. Run the same frozen cases against the buyer’s current checks and the candidate agent. Record incremental error classes, review time and the release decision affected. Keep holdouts separate from iterative prompt development.
What this method does—and does not—establish
A constructed case can test adherence to specified rules; it cannot establish a model’s real-world conversion rate, error prevalence or downstream revenue impact. A human panel can clarify ambiguous choices, but three reviewers are not a representative sample of the sales profession. A fictional example does not show that a buyer’s existing evaluation has this blind spot.
The question for a product owner is narrower: Did a controlled evaluation identify a consequential decision error the existing tests missed, and would that evidence change what ships?
Proposed design-partner study
Bring one decision your current eval cannot settle
PingPermit is developing a named-customer, evaluation-only GTM Judgment Sprint: 40 scenarios, three independent paid practitioners per scenario, 120 first-pass judgments and 12 repeat checks. The proposed price is $7,500, subject to agreed scope, qualified reviewers, permissions and verified funding. No studies have been represented as completed here.
Describe the decision, the important failure your evaluator may miss, and what release choice the evidence would inform. Start with a nonconfidential brief; no upload or signup is required.
Draft a nonconfidential evaluation briefOr email an inquiry. Sending a message does not grant a research or model-training license.
Context and sources
Unify’s June 22, 2026 account of its agent evaluation processdescribes overlapping reply labels, human-labeled examples and weighting action-relevant classes differently from raw accuracy. This establishes that sophisticated internal evaluations already exist, not that Unify needs this service or has any relationship with PingPermit. The NIST Generative AI Profile provides broader voluntary guidance on measurement and risk management; it does not certify this proposed rubric.