A convincing reply can hide incomplete work. Before an AI sales agent handles reminders or callbacks, test what happens outside the chat window.
The cases below use an illustrative appointment workflow. They are not customer results or a promise about a particular product.
Change the appointment after the reminder is queued
Confirm a visit, queue the reminder, then move the visit. The old reminder should no longer describe the old time as current.
Check the stored appointment revision and the pending action. The agent should not rely on an older conversation summary when current records disagree.
Revoke consent before the send happens
Give permission for a contact route, then withdraw it before execution. The queued action must re-check the latest state and stop.
An unknown permission should remain unknown. A confident interpretation of the buyer's tone is not evidence of consent.
Repeat the event without repeating the side effect
Deliver the same inbound event twice. Then simulate an action that succeeds at the provider but loses its response.
Inspect actual outgoing messages and records. The system needs a supported deduplication or reconciliation path, not simply an instruction to "avoid duplicates". Temporal's Activity guidance explains why retryable work must be designed for safe repetition.
Leave the assigned rep unavailable
Create an exception, assign a rep, then have the rep decline or fail to accept. Check that the task remains open and reaches a visible fallback.
Queued, assigned, accepted and completed should not collapse into one status. A buyer-facing callback promise must match the real owner and time.
Ask for a commitment outside the agent's permissions
Request a negotiated concession or a legal assurance. The agent should preserve the question and hand it to a responsible person, rather than inventing an exception from a similar document.
Application permissions should constrain the action even if the model produces a plausible answer. Logging a wrong commitment does not make it safe.
Compare outcomes across repeated tests
Run the same cases with varied wording and changed records. Count missed callbacks, duplicate actions, incorrect commitments and unclaimed tasks alongside speed and cost.
Agent-reliability research argues that one successful run is not a full reliability measure. It does not establish a sales-specific failure rate. A pilot still needs its own acceptance criteria.
Use flexible model reasoning for varied buyer input, with explicit workflow control around actions and completion. Keep the design small enough that the team can explain how it fails and who acts next.
Sources: Anthropic agent design guidance
Temporal Activity definition and idempotency
Agent reliability research
NIST Generative AI Profile