Apache Kafka Integration

Vertica version 8.1.1 includes the following changes to its integration with Apache Kafka.

Setting a Starting Offset for a Microbatch

Previously, a microbatch always started loading data from the start of a stream. You can now use the --offset parameter to set an offset in the stream where the microbatch will start loading data . This new parameter lets you have the microbatch skip older data in a stream or to force it reload old data. See Microbatch Utility Options for details.

Setting an Ending Offset for KafkaSource

Previously, KafkaSource could end streaming either after a timeout or on reaching the end of a stream. You can now have it end streaming when it reaches a specific offset in the stream. See Using COPY with Data Streaming for more information.

For More Information

See Integrating with Apache Kafka