Apache Kafka Integration

Changes to the Kafka Parser Functions

Vertica 9.1 introduces the following new features to KafkaAvroParser and KafkaJSONParsers:

• The Kafka JSON and Avro parser functions now have the same features as the Vertica Flex parsers. KafkaAvroParser now accepts the enforce_length and key_separator parameters, and KafkaJSONParser now accepts enforce_length, key_separator, suppress_nonalphanumeric_key_chars, and start_point_occurrence. See KafkaAvroParser and KafkaJSONParser for more information.
• The flattening behavior of the KafkaAvroParser parser has been changed to match that of the Flex parser.
• The KafkaSource UDL can now be canceled using one of the session closing functions such as CLOSE_ALL_SESSIONS.
• The Vertica installation package now installs the kafkacat utility in /opt/vertica/packages/kafka/bin. You can use this utility to troubleshoot issues transferring data between Vertica and Kafka. See Using kafkacat to Troubleshoot Kafka Integration Issues for more information.

Kafka Integration and Eon Mode

The Vertica integration with Apache Kafka now works in Eon Mode. There are several details you need to consider when streaming data from Kafka into an Eon Mode Vertica cluster. See Vertica Eon Mode and Kafka for details.