A/B/X testing with external models

You can use the External recommendations to include recommendations made by a third-party model in Synerise A/B/X tests.

Adding an external recommendation to tests

To use this feature:

1. Create a recommendation campaign and set the type to External.
   To learn how to create a recommendation campaign, see Creating recommendations.
2. Create an A/B/X test and select the above recommendation as one of the variants.
   To learn how to create an A/B/X test, see A/B/X testing.
3. When making a recommendation request, add the item IDs selected by the external model.
   • Those items must exist in the item feed used by the recommendation variant assigned to the profile.
   • The items are added in the query parameters of GET requests (externalItemId) or request body of POST requests (recommendedItemsFromExternalModel).
   • If you don’t add the external items and the external variant is assigned to the profile, no items are returned and a recommendation event isn’t generated. See the diagram below.
4. Monitor the results of the test as described in A/B/X testing.

Logic overview

This diagram explains how an A/B/X test response is processed when at least one of the variants is an external recommendation.

Logic description

From the perspective of sending the external items:

• If you send the external item(s) when you request an A/B/X test for a profile:
  1. A variant is assigned or retrieved for the profile.
  2. The response includes the recommended items and recommendation events are generated.
• If you don’t send the external item(s) when you request an A/B/X test for a profile:
  1. A variant is assigned or retrieved for the profile.
  2. If the selected variant is an external recommendation:
     1. The response doesn’t include any items (because they weren’t provided). The response includes information about the variant.
     2. A recommendation event isn’t generated.
     3. You must make another request, this time providing the items from the external model. These items must exist in the item feed associated with the assigned variant.
     4. The response includes the recommended items and you can display them.
     5. A recommendation event is generated.
  3. If the selected variant is a recommendation from a Synerise model:
     1. The response includes the recommended items and you can display them.
     2. A recommendation event is generated.