About Configuring a Data Load from S3

When you create an S3 Data Load using MC, you have the option of further configuring the job. You can optionally specify the following:

Add COPY Parameters

MC performs the loading job using a COPY statement. You can use the COPY Parameters field to further configure the COPY statement, or leave it blank. Only parameters used after the SOURCE parameter in a COPY statement are valid in this field.

Note that if you use the FILTER and PARSER parameters, they must appear in that order and precede any other parameters you use in the COPY Parameters field.

Specify EXCEPTIONS only if you set the Capture rejected data in a table field to No (see below).

In the following example, you could use the DELIMITER and SKIP parameters in the COPY parameters field to separate columns with a comma and skip 1 record of input data.

DELIMITER ',' SKIP 1

To add comments in the COPY statement using this field, begin your comment with a forward slash followed by an asterisk (/*) and end it with an asterisk followed by a forward slash (*/). Using a double hyphen (--) does not work in this field.

If COPY rejects the maximum number of rows, Vertica rolls back the target table rows without committing any data.

Capture Rejected Data in a Table

Use the Capture rejected data in a table field to create a rejected data table. A rejected data table allows you to view details about rejections through MC. If you set this field to Yes, you will be able to view rejected row data in the Load History tab.

You must have CREATE privilege on the schema if the table doesn't already exist. When you invoke multiple load processes for the same target table, MC appends all rejections data to the same table. MC generates a table using the following naming convention: schema.s3_load_rejections_target-table-name.

For more information about capturing rejected data, see Saving Rejected Data To a Table.

Set a Rejected Records Maximum

Use the Reject Max field to specify the maximum number of records that can be rejected before the load fails. If COPY rejects the maximum number of rows, Vertica rolls back the target table rows without committing any data.

Configuration Options

To further configure the loading job, you can use the following optional fields.

Field Details
COPY Parameters

Use the COPY Parameters field to further configure the COPY statement that will load your data.

Valid values:

  • FILTER
  • PARSER
  • DELIMITER
  • TRAILING NULLCOLS
  • NULL
  • ESCAPE (NO ESCAPE)
  • ENCLOSED
  • RECORD TERMINATOR
  • SKIP, SKIP BYTES
  • TRIM 'byte'
  • EXCEPTIONS 'path'
  • ENFORCELENGTH
  • ABORT ON ERROR
  • STORAGE

For more information about COPY syntax, see COPY and COPY Parameters.
Capture rejected data in a table

Use this field to create a rejected data table.

  • Yes: MC generates a rejected data table.
  • No: MC does not save rejected rows to a table.

For more information, see Saving Rejected Data To a Table.
Reject max Use this field to specify the maximum number of records that can be rejected before the load fails.