Sample Conversation Analyst Prompts

Overview

Below you'll find sample analyst prompts you can leverage. These prompts are just a starting point if you need help understanding how to build metrics.

📘

Tips & Tricks

Ensure you've set up your Prompt response type correctly to match the format of your response, if generating metrics for Agent Insights, you'll need to ensure you're using the Action response type.

Conversational Insight Prompts

Below is a list of sample prompts that can be used for gleaning insights from an entire conversation.

Escalation Appropriateness

Good for: Measuring whether AI and human agents are escalating at the right time, to the right place, with good context. Useful for improving AI Agent guardrails, routing logic, agent coaching, and handoff playbooks to reduce unnecessary transfers and “cold” escalations.

---

## Escalation Appropriateness

Review the full conversation transcript between one customer and one or more agents (AI and/or human). This prompt is only used when an escalation/transfer/handoff to another team or supervisor already occurred.

Your task is to determine whether the escalation was appropriate and well-handled—not whether the issue was ultimately resolved.

---

### What "appropriate escalation" means

Mark escalation as **appropriate** when it was:

- **Justified** (the issue reasonably required a supervisor or specialized team)
- **Timely** (not too early and not delayed after it became clear escalation was needed)
- **Correctly routed** (right team/person)
- **Warm handoff** (context was passed along so the customer didn't have to repeat everything)

Common justified reasons include: safety concerns, fraud/account takeover, legal/privacy requests, policy exceptions/discounts beyond authority, complex technical issues after reasonable troubleshooting, or explicit supervisor request.

---

### What makes escalation inappropriate

Mark escalation as **inappropriate** when it was:

- **Premature** (no real attempt to help, no basic troubleshooting)
- **Unnecessary** (simple/common request the agent should handle)
- **Poorly executed** (cold handoff, missing context, multiple transfers/bouncing)
- **Incorrectly routed** (wrong team/department)
  - **Avoidant** (agent escalates to dodge work they could do)

### Conversation transcript
Here is the converastion:

//INSERT CONVERSATION HISTORY BLOCK//

---

### Response Format

escalation appropriate: <yes|no|unclear>
issue: <brief description max ~15 words | n/a>
evidence: <1–2 sentences>

Knowledge Gap Analysis

**Good for: **Identifying where you need better documentation/training vs better system access/integrations. Useful for prioritizing KB updates, agent enablement, and API/CRM/OMS connector work based on real conversation failures.

## Knowledge Gap (with General vs Account-Specific)

Review the full conversation transcript between **one customer** and **one or more agents** (AI and/or human). Your task is to determine whether the conversation shows a **knowledge gap** that prevented the agent(s) from answering a reasonable, company-relevant question.

A **knowledge gap** means the customer asked something a representative *should* be able to answer using company knowledge (e.g., policy, process, product behavior, troubleshooting steps, pricing/billing rules, eligibility, timelines), but the agent(s) could not because the information was **missing, unclear, or inconsistent**.

### Guardrails (what NOT to flag)
Do **not** mark a knowledge gap for:
- AI Agent loops / getting stuck / repeating itself
- Routing or handoff issues
- Simple agent mistakes or misunderstandings  
Unless the underlying issue is truly that **company knowledge content is missing/unclear** (i.e., documentation would fix it).

### If a gap exists: classify the type
- **general**: A KB article/policy/process doc would answer it for most customers (no account lookup needed).
- **account specific**: Answer requires looking up customer/order/account details via internal systems or API access.
- **both**: The customer needs a general policy/process answer **and** an account-specific lookup/action.

### Response Format (return exactly these lines; no extra text)
knowledge gap: <yes|no|unclear>  
gap topic: <short label | not applicable>  
gap type: <general|account specific|both|not applicable>  
evidence: <1–2 sentences | not applicable>

Customer Effort Score (CES)

Good for: Measuring how hard customers have to work to get help, so you can pinpoint where conversations create friction (repetition, too many steps, unclear instructions, delays, transfers)

## Customer Effort Score (CES)

Review the full conversation transcript between one customer and one or more agents (AI and/or human). Your task is to determine the customer’s overall effort required to get help in this interaction. Focus on effort (friction, repetition, number of steps), not on whether the issue was ultimately resolved.

---

### What CES measures (effort, not satisfaction)
Score effort based on how hard the customer had to work to make progress, including:
- Number of back-and-forth turns
- Whether the customer had to repeat themselves or re-provide details
- Clarity of instructions and next steps
- Transfers/handoffs and whether context was preserved
- Time/delays or waiting for follow-up
- Workarounds required (calling elsewhere, emailing, visiting a store, etc.)
- Whether the agent caused unnecessary friction (irrelevant questions, loops, confusing guidance)

---

### CES Scale (choose one)

Mark **Very Low Effort** when:
- The customer’s request is handled quickly and clearly
- Minimal follow-up questions
- No repetition, no transfer friction

Mark **Low Effort** when:
- Small amount of back-and-forth or minor clarification needed
- The path to resolution is still straightforward

Mark **Moderate Effort** when:
- Noticeable friction: several turns, some repetition, or multi-step instructions
- The customer must do meaningful work, but progress is being made

Mark **High Effort** when:
- Substantial friction: repeated clarifications, multiple steps, delays, or transfer friction
- The customer struggles to get clear answers or forward progress

Mark **Very High Effort** when:
- Extreme friction: looping, multiple failed attempts, repeated transfers, or the customer is effectively blocked
- Little/no meaningful progress, or the customer must switch channels/escalate due to breakdowns

---

### Conversation transcript
Here is the conversation:
//INSERT CONVERSATION HISTORY BLOCK//

---

### Response Format

customer effort score: <Very Low Effort|Low Effort|Moderate Effort|High Effort|Very High Effort>  
reason/summary: <1 sentence, no PII>  
evidence: <1–2 sentences, no PII>  

Product / Service Feedback

Good for: Useful for capturing a lightweight voice-of-customer signal—it highlights which products/services are being mentioned, whether the sentiment is praise vs complaint vs suggestion, and what themes are emerging so teams can prioritize follow-up and improvement work beyond survey data alone.

Ask

Explain

## Product / Service Feedbac
Review the full conversation transcript between **one customer** and **one or more agents** (AI and/or human). Your task is to determine whether the customer provided **product or service feedback** (positive, negative, or suggestions) about the company’s offering—not just about the support interaction.

**Product/service feedback** includes opinions or experiences about the product/service itself (quality, features, usability, performance, reliability, value, delivery experience, on-site experience, etc.), including suggestions for improvement.

### Guardrails (what NOT to flag)
Do **not** mark product/service feedback for:
- Pure **support satisfaction** (“thanks for your help”, “agent was great”) unless the customer also comments on the product/service itself
- Neutral factual statements without evaluation (e.g., “my package arrived yesterday”)
- Questions that are only information-seeking unless they include an evaluation

### If feedback exists: classify the type
- **complaint**: negative experience or dissatisfaction with the product/service
- **praise**: positive experience or endorsement
- **suggestion**: explicit improvement idea/request (may be positive or negative context)
- **mixed**: both positive and negative feedback in the same conversation

### Response Format (return exactly these lines; no extra text)
product/service feedback: <yes|no|unclear>  
product/service name: <name mentioned | unknown | not applicable>  
feedback type: <complaint|praise|suggestion|mixed|not applicable>  
feedback topic: <short label | not applicable>  
evidence: <1–2 sentences | not applicable>

**Field notes**
- `product/service name`: Return the specific product/service name the customer refers to (e.g., plan name, feature name, service offering, product model). If feedback is present but no name is given, use `unknown`. If no feedback, use `not applicable`.

**Good for:** Capturing VOC insights from support conversations with enough specificity (which product/service) to route to the right owners and quantify recurring feedback themes.

Human Agent Scoring

Below are some sample agent scoring prompts to help you get started.

📘

In order for your agent metrics to show up in Agent Insights, you'll need to ensure you've followed the instructions in the tracking human agent metrics section.

Financial Explanation Clarity (Charges / Credits / Refunds)

Good for: Billing confusion is a major driver of repeat contacts and disputes; this flags where agents need better scripting, training, or tooling to set expectations clearly.

**Overview:**  
Your job is to analyze a conversation transcript that represents a customer service interaction between a single customer and one or more agents.

**Summary:**  
Specifically, you need to: **Determine whether billing-related charges/credits/refunds were clearly explained when they were discussed.**  
If charges/credits/refunds are **not discussed**, leave the result **blank**.

**Terminology:**

- **Charges explanation** includes explaining one or more of: charges, authorization vs charge, credits, refunds, refund timing, partial refunds, fees, tax, billing corrections, and what the customer should expect next.
- **Clearly explained** means the explanation is understandable and includes the key expectations a reasonable customer needs (what will happen + when/next step if stated).
- **Poorly explained** means the explanation is missing key details, confusing, contradictory, or the customer remains unclear and asks again.

**Analysis Instructions:**

1. **Read the entire transcript thoroughly**
  - Do not make assumptions based on initial statements.
  - Remember that you are judging the conversation in its entirety. Consider all messages between the customer and the agents.
  - Each transcript message will be formatted like so: `<<AGENT ID>>: <<MESSAGE TEXT>>`
2. **Check whether charges/credits/refunds are discussed**
  - If they are not discussed in any meaningful way, output a blank value for `explained_charges` and `reason`.
3. **Score explanation quality (only if discussed)**
  - If discussed, score whether the explanation was clearly sufficient vs clearly insufficient, based only on the transcript.

---

**Conversation Transcript**:  
Here is the conversation transcript you will be considering for your task:  
//INSERT CONVRESATION HISTORY BLOCK//

**Agent IDs**:  
Here is the list of agents IDs you will be considering for your task:  
//INSERT conversation.allAgentIDS FIELD //

**Response Format (return exactly these lines; no extra text)**  
explained_charges: <Clearly explained charges|Poorly explained charges|>  
reason: <1–2 sentences|>