Hi, I'm Ask INFA!
What would you like to know?
ASK INFAPreview
Please to access Ask INFA.
Not Found
English

Page Not Found

The page or file you are looking for has been moved or no longer exists.

The following links might help you find the content you need:


You can also search the Documentation Portal or visit the home page.

English
  • Data Integration Connectors
  • All Products

Table of Contents

Search

  1. Introduction to Amazon S3 V2 Connector
  2. Amazon S3 V2 connections
  3. Amazon S3 V2 sources and targets
  4. Mappings and mapping tasks with Amazon S3 V2
  5. Migrating a mapping
  6. Upgrading to Amazon S3 V2 Connector
  7. Data type reference
  8. Troubleshooting

Amazon S3 V2 Connector

Amazon S3 V2 Connector

Back Next

Amazon Redshift manifest file

Amazon Redshift manifest file

You can use an Amazon Redshift manifest file created by the UNLOAD command to read multiple flat files from Amazon S3. All flat files must have the same metadata and must be available in the same Amazon S3 bucket.
Create a
.manifest
 file and list all the source files with the URL that includes the bucket name and full object path for the file. You must specify the
.manifest
 file name in the following format:
<file_name>.manifest
.
For example, the Amazon Redshift manifest file contains source files in the following format: 
{
    "entries": [
            {"url": "s3://mybucket-alpha/2013-10-04-custdata", "mandatory":true},
            {"url": "s3://mybucket-alpha/2013-10-05-custdata", "mandatory":true},
            {"url": "s3://mybucket-beta/2013-10-04-custdata", "mandatory":true},
            {"url": "s3://mybucket-beta/2013-10-05-custdata", "mandatory":true},
    ]
}
The Redshift manifest file format contains the following tags:
url
The url tag consists of the source file in the following format:
"url": "<endpoint name>://<folder path>/<filename>", "mandatory":<value>
mandatory
Amazon S3 V2 Connector uses the
mandatory
 tag to determine whether to continue reading the files in the
.manifest
 file or not, based on the following scenarios:
  • If the value of
    mandatory
     tag is
    true
    , and the S3 bucket does not have the specified source file, Amazon S3 V2 Connector does not read the rest of the files as well in the
    .manifest
     file. The mapping task fails.
  • If the value of
    mandatory
     tag is
    false
    , and the S3 bucket does not have the specified file, Amazon S3 V2 Connector continues to read the rest of the files in the
    .manifest
     file in a sequence.
  • If the
    .manifest
     file does not contain any files, the mapping task fails.
By default, the value of
mandatory
 tag is
false
.
0 COMMENTS
We’d like to hear from you!