Using Kafka Event Logger with History Node

You can use Apache Kafka as a transport for Interaction Server reporting events. It is especially useful if you plan to use Interaction Server Kafka Capture Point, allowing you not to install a separate JMS broker. The same Kafka installation can be used as a transport both for Capture Point and Event Logger needs.

Prerequisites

You can choose either:

1. Import template from the iwd_history/event_logger_templates folder as described in the iWD History Node Application Definition topic.
2. Configure Interaction Server Kafka Event Logger from scratch as described in the Using the Groovy Event Logger with Apache Kafka topic in the eServices Administration Guide.

Kafka Settings

For iWD History Node to be able to consume data written by Interaction Server, set the following option in the Kafka Event Logger Application Object that is used by these iWD History Node and Interaction Server respectively:

[kafka-settings]\binary-output = true

Consumer Options Description

• [consumer-options]\poll-delay—The maximum time in milliseconds to block the consumer when it polls for records.
• [consumer-options]\max.poll.records—The maximum number of records returned in a single call to poll().

Important

1. max.poll.records and poll-delay values can be tuned for better performance depending on your environment and the size of interaction user data. Please change these values only if you understand what you are doing and why.
2. Option [iWD]\batch-size is not taken into account when Kafka Event Logger is used. Use option [consumer-options]\max.poll.records instead.

All other options are standard Kafka consumer options and can be found in the official Kafka documentation.

Additional Considerations

iWD History Node configured to use Kafka Event Logger must use the same Kafka deployment and its configurations once data (events) are being produced and written by Interaction Server to Kafka and the same are consumed by iWD History Node.

Important

If at any point, offsets in Kafka are reset or a different deployment of Kafka is being used then use a different topic for Event Logger. In this scenario, you must also update the Interaction Server configuration. iWD History Node will skip all the events up to the latest offset number that it has consumed prior to reset to safeguard itself against possible duplicated data consumption.