Mass Ingestion

Back Next

Guidelines for Microsoft Azure Synapse Analytics targets

Consider the following guidelines when you use Microsoft Azure Synapse Analytics targets:

To deploy and run an

application ingestion

task with a Microsoft Azure Synapse Analytics target, the target connection must specify a database user who has the CONTROL permission on the target database. To grant the CONTROL permission to the user, use the following SQL statements:

USE database_name; 
GRANT CONTROL TO user_name;

The CONTROL permission is required for initial load, incremental load, and combined initial and incremental load jobs. The permission allows

Mass Ingestion Applications

to create target tables and database objects such as external data source, external file format, and database scoped credential objects if they do not exist in the database. The CONTROL permission is specifically required for creating external data source and database scoped credential objects.

You must manually create the master key. To create the master key, you must have the CONTROL permission on the database.

Application ingestion

jobs first send data to a Microsoft Azure Data Lake Storage Gen2 staging file before writing the data to Microsoft Azure Synapse Analytics target tables. The staging file uses the hexadecimal x1d separator as the field delimiter. After the data is written to the target, the data stored in the table-specific directory that includes the staging files are deleted.

If you use Microsoft Azure Data Lake Storage Gen2 with a Microsoft Azure Synapse Analytics connection, you must enable the

Hierarchical namespace

option in Microsoft Azure Data Lake Storage. With this setting, blob storage is not recommended.

When you configure an

application ingestion

task for a Microsoft Azure Synapse Analytics target, ensure that each source object that you select for replication meets the following criteria:

The object must not contain more than 1024 fields and the size of each field must be less than 500 KB.

The object must not contain any record that is greater than 1 MB in size.

The object must not contain more than 32 primary keys.

The primary keys of the object must be of a data type that Microsoft Azure Synapse Analytics supports for primary keys.

Incremental load jobs and combined initial and incremental load jobs generate a recovery table named INFORMATICA_CDC_RECOVERY on the target to store internal service information. The data in the recovery table prevents the jobs that are restarted after a failure from propagating previously processed data again. The recovery table is generated in the schema of the target tables.

After an

application ingestion

job loads data to a Microsoft Azure Synapse Analytics target by using external tables, the job does not drop the log tables and external tables created on the target, even though these tables might be re-created when the job starts again.

Application ingestion jobs configured for Microsoft Azure Synapse Analytics targets do not replicate the renaming of source fields on the target.

Rename Saved Search

Table of Contents

Mass Ingestion

Mass Ingestion

Guidelines for Microsoft Azure Synapse Analytics targets

Guidelines for Microsoft Azure Synapse Analytics targets