Hi, I'm Ask INFA!
What would you like to know?
ASK INFAPreview
Please to access Bolo.

Table of Contents

Search

  1. Introducing Mass Ingestion
  2. Getting Started with Mass Ingestion
  3. Connectors and Connections
  4. Mass Ingestion Applications
  5. Mass Ingestion Databases
  6. Mass Ingestion Files
  7. Mass Ingestion Streaming
  8. Monitoring Mass Ingestion Jobs
  9. Asset Management
  10. Troubleshooting

Mass Ingestion

Mass Ingestion

Handling source schema changes

Handling source schema changes

You can configure
Mass Ingestion Applications
to automatically detect source schema changes, also called
schema drift
, and handle these changes on the target. This feature is available only for the incremental load and combined initial and incremental load tasks.
Mass Ingestion Applications
can detect the following types of source schema changes:
  • Add field
  • Modify field
  • Drop field
  • Rename field
When you configure an
application ingestion
task, on the
Schedule and Runtime Options
page of the
application ingestion
task wizard, you can specify the types of source schema changes that
Mass Ingestion Applications
must propagate for the job associated with the task. You can also specify how the job must handle each type of source schema change. For example, you can configure the task to ignore the changes, replicate them, or stop the job when a particular type of schema change occurs on the source. For more information, see Configuring schedule and runtime options.
Mass Ingestion Applications
detects a schema change in a source object only after Data Manipulation Language (DML) operations occur on the altered source object. If multiple schema changes occur without intervening DML operations,
Mass Ingestion Applications
detects all the schema changes together when a DML operation occurs.
  • Application ingestion
    jobs do not detect the schema changes that occur on the source after you deploy a job and before the first run of the job.
  • Application ingestion
    jobs do not replicate source changes that add, remove, or modify primary key or unique key constraints. If these types of changes occur on the source, you must re-synchronize the target tables.
  • Mass Ingestion Applications
    does not detect the addition and renaming of fields in the schema of Microsoft Dynamics 365 sources.
  • The
    application ingestion
    jobs configured for Microsoft Azure Synapse Analytics targets do not replicate the renaming of source fields on the target.
  • The
    application ingestion
    jobs configured for Google BigQuery targets do not replicate the modification and renaming of source fields on the target.

0 COMMENTS

We’d like to hear from you!