KafkaExport

Outputs rows that Vertica was unable to send to Kafka. If all of your messages have imported successfully, this function returns zero rows. You can use this function to copy failed messages to a secondary table for evaluation and reprocessing.

Syntax

KafkaExport(partitionColumn, keyColumn, valueColumn USING PARAMETERS brokers='host', source='sourcename')
OVER () FROM table

Parameters

Argument Description
partitionColumn Optional. The target partition for the export. If you do not specify a partition, Vertica uses the default partition. You can use the partition column to send messages to partitions that map to Vertica segments. Specify a value of NULL to skip this parameter.
keyColumn Optional. The user defined key value associated with the valueColumn. Specify a value of NULL to skip this parameter.
valueColumn Optional. The message itself. The column must have a data type of VARCHAR. Specify a value of NULL to skip this parameter.
brokers A list of one or more hosts.
source The source to which you are exporting.

Examples

=> SELECT KafkaExport(partion, messageId, message USING PARAMETERS brokers='KafkaBroker'
, source='failure_test',
                      kafka_conf='message.max.bytes=64000')
     OVER (PARTITION BEST)
                FROM failure_test;		
 partition |     key     |                       substr               |         failure_reason
-----------+-------------+--------------------------------------------+-------------------------
      -123 | key1        | negative partition not allowed             | Local: Unknown partition
     54321 |             | nonexistant partition                      | Local: Unknown partition
         0 | normal key1 | normal value1                              | Broker: Message size too large
         0 |             | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa | Broker: Message size too large
         0 | normal key2 | normal value2                              | Broker: Message size too large