Accessing VERTICA_WRITER Plug-in Attributes

  1. While in Workflow Manager, double-click any task.
  2. In the Edit Tasks dialog box, select the Mappings tab.
  3. Select your source qualifier (this example uses datatypes_tgt).
  4. Minimize the Readers and Connections areas to focus on Properties.

    Plug-in features are listed under the Properties area.

The table that follows lists and describes the plug-in attributes for target/writer.

Attribute Value

Schema Name

Change the schema name by entering a new name in this field.

 

Truncate Target Table

Select this option if you have a workflow that should truncate its targeted table before loading data.

Pre SQL

Enter complete SQL statements that run before you write to a table.

Post SQL

Enter complete SQL statements that run after you write to a table.

Target Table Name

Change the target table name by entering a new name in this field.

Copy Direct When selected, writes directly to the ROS container. More efficient for bulk loading.
Reject file directory

Reject file directory and Reject file name work in tandem to record rejected rows. In this field you specify the directory path for the reject file.

Reject file name

Specify the name of the file that holds the rejected rows.

There is one log file per partition. If there are multiple log files, a number is appended to the file names. For example, rejects.txt would become rejects_01.txt and rejects_02.txt.

Insert If selected, makes insert the update strategy for the target.
Delete If selected, makes delete the update strategy for the target.
Update List box offers standard update options for target.
Time UTC Conversion

If selected, keeps time synchronized when you are using Vertica as both source and target. Time data changes to UTC time zone.

Select this option and the reader option only when using Vertica as both source and target.

If unchecked, when you are using Vertica as both source and target, time data changes to the JVM time zone.

Copy Local Method

Choose method from list box.

None. The default; in this case, no local copy method is used to stream data.

Important: To take advantage of increased performance of EnableStreamingBatchInsert, you must set Copy Local Method to None.

Delimited. Creates a stream with pipes and newline delimiters. This method fails if there are pipes or newline delimiters in the data you are transmitting.

Native Varchar Converts to new format. Use if data is mostly some form of strings.

Native binary Serializes java objects to Vertica objects.

Target Number

Default behavior. The Target Number default is zero. If you leave the default (or set Target Number to any number less than or equal to 1), the plug-in uses the IP address you specify in the Connection String when configuring workflow connections. See Configuring Your Workflow Connections for more information on where you specify the IP address.

In regards to load balancing, the Target Number default operates as follows.

  • If the IP Address you specified in the Connection String is not an Vertica node, the plug-in targets that IP Address. Any load balancing policy enabled on the IP Address is used.
  • If the node you specified in the Connection String is an Vertica node, note the following:

    Vertica native connection load balancing is off by default; in this case, the plug-in uses the IP address you specify in the connection string.

    If Vertica load balancing is enabled, then the plug-in targets nodes as defined by the load balancing scheme on the Vertica node.

Non-default behavior. You override the plug-in’s default behavior by setting the option Target Number to a value greater than 1 (up to 16). When the value of the Target Number is set to greater than 1, the plug-in targets that number of Vertica nodes and uses ROUNDROBIN as its load balancing scheme. This overrides the native Vertica load balancing scheme.

For information on native connection load balancing, refer to, About About Native Connection Load Balancing in the Administrator’s Guide. For information on setting a load balancing policy on a Vertica server, refer to SET_LOAD_BALANCE_POLICY in the SQL Reference Manual.