Measuring the Customer Agent

IMPORTANT: This feature is available in private preview mode and accessible only on selected workspaces. To get access, contact your account manager or Synerise Support.

Every conversation with the agent is recorded as events on the customer's profile, in the same way as page visits or transactions. You can use them in segmentations, metrics, funnels, and automation, and combine them with the rest of the profile's history. This article explains what each event tells you, how to analyze conversations in Synerise, and how to pass information to your own analytics tools.

The parameters of each event are listed in Customer Agent events.

Events at a glance

Event When it is generated What it tells you
assistant.conversationStart When the chat is initialized on a page and a conversation begins How many conversations started, on which channel (WEB_DESKTOP, MOBILEWEB, MOBILE_APP), and with which agent (assistantId)
assistant.responseGenerated Every time the agent answers a customer's message What the agent did (action), whether it succeeded (status, errorReason), whether guardrails blocked it (guardrails), which items it returned (items), and the position of the answer in the conversation (seqNo)
assistant.click When the customer clicks a product card or a product link in an answer Which item was clicked (itemId), from which element (clickedFromComponent: CAROUSEL for a product card, BUBBLE_TEXT or TEXT for a link in the text), and which answer it came from (correlationId, seqNo)
assistant.responseFeedback When the customer rates an answer with thumbs up or thumbs down Which answer was rated (seqNo), how (vote: POSITIVE or NEGATIVE), and the optional comment (comment)

All four events share threadId, the ID of the conversation, so you can follow a single conversation from start to click.

IMPORTANT: A conversation starts when the chat is initialized, not when the customer opens it. If your page initializes the chat on every page view, an assistant.conversationStart event is generated for every view, and the work of preparing the greeting is counted in the eventWeight parameter of the event. If you want the event to mean "the customer opened the chat", initialize the chat when the launcher is clicked for the first time:

let chat = null;
launcher.addEventListener("click", () => {
  if (!chat) {
    chat = init({ rootElementId: "synerise-assistant", stream: true });
  }
  chat.open();
});

In this variant the launcher is always visible, because the chat can only check whether the agent is available once it is initialized.

What the action values mean

The action parameter of assistant.responseGenerated describes how the agent produced an answer. Typical meanings:

Value The agent…
search searched the AI Search index for products
recommend used a recommendation campaign connected as a tool
brickworks fetched data through a Brickworks schema connected as a tool
mcp_tool called a tool on an MCP server
answer_products answered a question about products already in the conversation
clarify asked the customer a follow-up question before searching
button_action reacted to a button or suggestion the customer tapped
link returned a link
message replied with a plain conversational message
reject declined the request, for example because it was outside its job or blocked by guardrails

The share of clarify answers shows how often the agent needs more information, and the share of reject answers shows how often customers ask for things the agent is not meant to handle. Both are useful when you tune the Assistant job, Engagement level, and Suggestions in the agent configuration.

Analyzing conversations in Synerise

On a profile

Open a customer's profile and review the activity. Each conversation appears as an assistant.conversationStart event followed by assistant.responseGenerated events, one per answer, and assistant.click events for the products the customer opened. This is the quickest way to verify a new implementation and to investigate a single customer's experience.

In segmentations

Build segmentations such as:

  • Profiles who started a conversation with the agent in the last 30 days.
  • Profiles who clicked a product recommended by the agent but did not buy.
  • Profiles whose requests were rejected (assistant.responseGenerated with action equal to reject).

Use them to target follow-up campaigns or to exclude customers who already used the agent from campaigns that promote it.

In metrics and reports

Typical measures built from the events, for example in reports:

Measure How to calculate it
Conversations Count of assistant.conversationStart
Engaged conversations Count of distinct threadId in assistant.responseGenerated
Answers per conversation assistant.responseGenerated divided by engaged conversations
Click-through assistant.click divided by assistant.responseGenerated with a non-empty items list
Purchase after conversation A funnel: assistant.conversationStart, then assistant.click, then a purchase event
Rejections Share of assistant.responseGenerated with action equal to reject

Compare the measures by channel to see how the chat performs on desktop, mobile web, and in the app, and by assistantId if you run more than one agent.

NOTE: The default retention of the agent events is 30 days. If you need longer trends, check the retention settings of these events before you rely on them.

Attributing clicks to their source

Each product returned by the agent carries the IDs that tie it to its source. They are copied to the assistant.click event:

  • correlationId links the click to the answer that contained the product.
  • campaignId is set when the product came from a recommendation campaign connected as a tool. It lets you compare the agent's clicks with the campaign's other placements.
  • searchCorrelationId is set when the product came from AI Search.

Response ratings

If response feedback is enabled in the agent configuration, customers can rate each answer with thumbs up or down and add a comment after a thumbs down. The rating and the comment are stored with the answer in the conversation. Use them to review conversations with negative feedback and to adjust the agent's instructions.

Each rating is also recorded as an assistant.responseFeedback event, so you can analyze ratings in Synerise, for example to track the share of negative ratings over time or to build a segment of customers who rated an answer negatively.

Connecting your own analytics tools

If you use a web analytics tool or a tag manager, the chat exposes callbacks that you can forward to it:

Callback When it runs Useful for
onLoad Once, when the chat learns whether the agent is available Counting how often the launcher was shown
onMessage After every answer, with the conversation ID and the messages Counting answers, tracking the first answer of a conversation
onStreamError When the agent refuses a request (streaming only), with the reason (type) and the side that was refused (stage) Measuring refusals separately from technical errors
onConversationTitle When Synerise generates the conversation's title (streaming only) Labeling the conversation in your own tools
init({
  rootElementId: "synerise-assistant",
  stream: true,
  onLoad: ({ assistantVisible }) => {
    window.dataLayer?.push({ event: "assistant_available", available: assistantVisible });
  },
  onMessage: ({ meta, data }) => {
    window.dataLayer?.push({ event: "assistant_answer", threadId: meta.threadId, messages: data.messages.length });
  },
  onStreamError: ({ type, stage }) => {
    window.dataLayer?.push({ event: "assistant_refused", type, stage });
  },
});

Product clicks are recorded by Synerise automatically. To send them to another tool as well, developers can wrap product cards with the product card slot; see Slots in the SDK reference.

Custom actions

A Custom action tool in the agent configuration lets the agent return a named action - for example open the size guide or add to cart - instead of text. Your website decides what happens next. Handling custom actions requires the React or Preact package (onCustomAction); the plain init() function does not expose them yet. See Customer Agent SDK reference.

What's next

Canonical URL: https://hub.synerise.com/docs/ai-hub/customer-agent/measuring-the-customer-agent