Message Queues logic

The Integration module uses Magento’s Message Queue Framework (MQF) to send data to Synerise system. This solution is meant to use RabbitMQ as a message broker, but also provides a MySQL implementation.

The Integration plugin will use RabbitMQ (if available) or fall back to MySQL based on general application configuration.

It’s highly recommended to set up aqmp for better performance.

AMQP Configuration

To install and set up RabbitMQ, refer to Magento Documentation. Additionally, from 2.4.3 Magento version, Amazon offers cloud service for AWS MQ.

Queues

The Integration module will setup a set of general queues as well as some specific to configured data models and enabled stores. Queues based on configuration are generated by running the bin/magento setup:upgrade command.

Consumers definition

Consumers are generated dynamically based on your configuration of enabled stores and models. Changing those options requires running the bin/magento setup:upgrade command afterwards.

To verify the list of consumers, run the bin/magento queue:consumers:list command.

Optional setup for consumers

Magento allows an additional, optional configuration of consumers to be run. If you defined a custom consumer array to be run, as in documentation, please make sure to add Synerise consumers to the list. To learn more, refer to Magento documetation.

Starting consumers

Consumers can be started from the command line. By default, the process is started by cron.

Refer to Magento Documentation to learn more about possible options.

Debugging

In case of issues with event or data processing, you can try to follow the logic behind the queues to check if all of their stages are executed properly.

1. In case of events, check if queues are enabled (more information here).
2. To enable triggering customer events, enable tracking.
3. Make sure that synchronization is enabled for the models and stores that are to be synchronized
4. Temporarily enable request logging to see if requests are sent.
5. For debugging, it’s recommended to disable crons and start consumers manually.
6. Trigger an action that will populate the appropriate queue with a message to be processed:
   • Perform an action that produces an event (for example, adding an item to the cart)
   • Continuous synchronization
   • Full synchronization - In this case, there is an intermediate queue (synerise.queue.data.scheduler) that schedules synchronization.
   • Batch synchronization

Debugging RabbitMQ setup

1. Log in to the RabbitMQ website.
2. Go to the Queues tab.
3. Find a queue of the topic you want to debug topic (refer to the table in the Queues section of this article) and see if the number of messages ready to process has risen. If the consumer is running in background, the message might be processed almost immediately.
4. For debugging, it’s recommended to disable crons and start consumers manually.
5. When a consumer picks a message from the queue, a log with the request and response is produced and the number of ready message decreases.
6. In Synerise, on the profile of a test user, check if the event was generated or if data from other modules (products, orders, subscriptions, and so on) are available in Synerise.

Debugging MySQL setup

1. Log in to your MySQL interface.
2. Check if the queue_message and queue_message_status tables get populated with messages.
3. When a consumer picks a message from the queue, a log with the request and response is produced and if the message status is set to 4, this means successful processing.
4. In Synerise, on the profile of a test user, check if the event was generated or if data from other modules (products, orders, subscriptions, and so on) are available in Synerise.