Measuring the Customer AI Assistant

IMPORTANT: This feature is available in private preview mode and accessible only on selected workspaces. To get access, contact your account manager or Synerise Support.

Every conversation with the assistant is recorded as events on the customer's profile, in the same way as page visits or transactions. You can use them in segmentations, metrics, funnels, and automation, and combine them with the rest of the profile's history. This article explains what each event tells you, how to analyze conversations in Synerise, and how to pass information to your own analytics tools.

The parameters of each event are listed in Customer AI Assistant events.

Events at a glance

Event When it is generated What it tells you
assistant.conversationStart When the chat is initialized on a page and a conversation begins How many conversations started, on which channel (WEB_DESKTOP, MOBILEWEB, MOBILE_APP), and with which assistant (assistantId)
assistant.responseGenerated Every time the assistant answers a customer's message What the assistant did (action), whether it succeeded (status, errorReason), whether guardrails blocked it (guardrails), which items it returned (items), and the position of the answer in the conversation (seqNo)
assistant.click When the customer clicks a product card or a product link in an answer Which item was clicked (itemId), from which element (clickedFromComponent: CAROUSEL for a product card, BUBBLE_TEXT or TEXT for a link in the text), and which answer it came from (correlationId, seqNo)

All three events share threadId, the ID of the conversation, so you can follow a single conversation from start to click.

IMPORTANT: A conversation starts when the chat is initialized, not when the customer opens it. If your page initializes the chat on every page view, an assistant.conversationStart event is generated for every view, and the work of preparing the greeting is counted in the eventWeight parameter of the event. If you want the event to mean "the customer opened the chat", initialize the chat when the launcher is clicked for the first time:

let chat = null;
launcher.addEventListener("click", () => {
  if (!chat) {
    chat = init({ rootElementId: "synerise-assistant", stream: true });
  }
  chat.open();
});

In this variant the launcher is always visible, because the chat can only check whether the assistant is available once it is initialized.

What the action values mean

The action parameter of assistant.responseGenerated describes how the assistant produced an answer. Typical meanings:

Value The assistant…
search searched the AI Search index for products
recommend used a recommendation campaign connected as a tool
brickworks fetched data through a Brickworks schema connected as a tool
mcp_tool called a tool on an MCP server
answer_products answered a question about products already in the conversation
clarify asked the customer a follow-up question before searching
button_action reacted to a button or suggestion the customer tapped
link returned a link
message replied with a plain conversational message
reject declined the request, for example because it was outside its job or blocked by guardrails

The share of clarify answers shows how often the assistant needs more information, and the share of reject answers shows how often customers ask for things the assistant is not meant to handle. Both are useful when you tune the Assistant job, Engagement level, and Suggestions in the assistant configuration.

Analyzing conversations in Synerise

On a profile

Open a customer's profile and review the activity. Each conversation appears as an assistant.conversationStart event followed by assistant.responseGenerated events, one per answer, and assistant.click events for the products the customer opened. This is the quickest way to verify a new implementation and to investigate a single customer's experience.

In segmentations

Build segmentations such as:

  • Profiles who started a conversation with the assistant in the last 30 days.
  • Profiles who clicked a product recommended by the assistant but did not buy.
  • Profiles whose requests were rejected (assistant.responseGenerated with action equal to reject).

Use them to target follow-up campaigns or to exclude customers who already used the assistant from campaigns that promote it.

In metrics and reports

Typical measures built from the events, for example in reports:

Measure How to calculate it
Conversations Count of assistant.conversationStart
Engaged conversations Count of distinct threadId in assistant.responseGenerated
Answers per conversation assistant.responseGenerated divided by engaged conversations
Click-through assistant.click divided by assistant.responseGenerated with a non-empty items list
Purchase after conversation A funnel: assistant.conversationStart, then assistant.click, then a purchase event
Rejections Share of assistant.responseGenerated with action equal to reject

Compare the measures by channel to see how the chat performs on desktop, mobile web, and in the app, and by assistantId if you run more than one assistant.

NOTE: The default retention of the assistant events is 30 days. If you need longer trends, check the retention settings of these events before you rely on them.

Attributing clicks to their source

Each product returned by the assistant carries the IDs that tie it to its source. They are copied to the assistant.click event:

  • correlationId links the click to the answer that contained the product.
  • campaignId is set when the product came from a recommendation campaign connected as a tool. It lets you compare the assistant's clicks with the campaign's other placements.
  • searchCorrelationId is set when the product came from AI Search.

Response ratings

If response feedback is enabled in the assistant configuration, customers can rate each answer with thumbs up or down and add a comment after a thumbs down. The rating and the comment are stored with the answer in the conversation. Use them to review conversations with negative feedback and to adjust the assistant's instructions.

Connecting your own analytics tools

If you use a web analytics tool or a tag manager, the chat exposes callbacks that you can forward to it:

Callback When it runs Useful for
onLoad Once, when the chat learns whether the assistant is available Counting how often the launcher was shown
onMessage After every answer, with the conversation ID and the messages Counting answers, tracking the first answer of a conversation
onStreamError When the assistant refuses a request (streaming only), with the reason (type) and the side that was refused (stage) Measuring refusals separately from technical errors
onConversationTitle When Synerise generates the conversation's title (streaming only) Labeling the conversation in your own tools
init({
  rootElementId: "synerise-assistant",
  stream: true,
  onLoad: ({ assistantVisible }) => {
    window.dataLayer?.push({ event: "assistant_available", available: assistantVisible });
  },
  onMessage: ({ meta, data }) => {
    window.dataLayer?.push({ event: "assistant_answer", threadId: meta.threadId, messages: data.messages.length });
  },
  onStreamError: ({ type, stage }) => {
    window.dataLayer?.push({ event: "assistant_refused", type, stage });
  },
});

Product clicks are recorded by Synerise automatically. To send them to another tool as well, developers can wrap product cards with the product card slot; see Slots in the SDK reference.

Custom actions

A Custom action tool in the assistant configuration lets the assistant return a named action - for example open the size guide or add to cart - instead of text. Your website decides what happens next. Handling custom actions requires the React or Preact package (onCustomAction); the plain init() function does not expose them yet. See AI Assistant SDK reference.

What's next

Canonical URL: https://hub.synerise.com/docs/ai-hub/ai-assistant/measuring-the-ai-assistant