Hi, I'm Ask INFA!
What would you like to know?
ASK INFAPreview
Please to access Bolo.

Table of Contents

Search

  1. Introducing Mass Ingestion
  2. Getting Started with Mass Ingestion
  3. Connectors and Connections
  4. Mass Ingestion Applications
  5. Mass Ingestion Databases
  6. Mass Ingestion Files
  7. Mass Ingestion Streaming
  8. Monitoring Mass Ingestion Jobs
  9. Asset Management
  10. Troubleshooting

Mass Ingestion

Mass Ingestion

Snowflake target properties

Snowflake target properties

When you define an
application ingestion
task, you must specify the properties for your Snowflake target on the
Target
page of the task wizard.
The following table describes the Snowflake target properties that appear in
Target
section:
Property
Description
Target Creation
The only available option is
Create Target Tables
, which generates the target tables based on the source objects.
Schema
The target schema in which
Mass Ingestion Applications
creates the target tables.
Stage
The name of internal staging area that holds the data read from the source before the data is written to the target tables. The name must not include spaces. If the staging area does not exist, it will be automatically created.
Apply Mode
For incremental load and combined initial and incremental load jobs with Snowflake targets, indicates how source DML changes, including inserts, updates, and deletes, are applied to the target. Options are:
  • Standard
    . Accumulate the changes in a single apply cycle and intelligently merge them into fewer SQL statements before applying them to the target. For example, if an update followed by a delete occurs on the source row, no row is applied to the target. If multiple updates occur on the same column or field, only the last update is applied to the target. If multiple updates occur on different columns or fields, the updates are merged into a single update record before being applied to the target.
  • Soft Delete
    . Apply source delete operations to the target as soft deletes. A soft delete marks the deleted row as deleted without actually removing it from the database. For example, a delete on the source results in a change record on the target with "D" displayed in the INFA_OPERATION_TYPE column. If an update followed by a delete occurs on the source, two records are written to the target both with "D" displayed in the INFA_OPERATION_TYPE column.
    Consider using soft deletes if you have a long-running business process that needs the soft-deleted data to finish processing, to restore data after an accidental delete operation, or to track deleted values for audit purposes.
  • Audit
    . For Snowflake targets only, ingest change data into an audit table on the target system by using insert operations, instead of merging and applying the changes to the target database. Consider using audit tables if you want to perform computations or other downstream processing on the data before applying it to the target database or if you want to examine the changes. You can add metadata columns for SQL change operations to the audit table by setting options under the
    Advanced
    section.
Default is Standard.

0 COMMENTS

We’d like to hear from you!