This release includes enhancements to the following connectors.
Adabas CDC Connector
When you restart a task that uses a CDC connector to connect to the source, you can now choose whether to perform a cold start or warm start from the job details page. A cold start restarts the task from the beginning, without recovery. When you cold start a failed task that has recovery enabled, Cloud Data Integration does not process the recovery data. A warm start resumes processing from the point where the task ended, with recovery. Typically, if a task fails after an initial cold start, a warm start is used to restart the task with recovery.
The cold start option is not supported for tasks that have Snowflake targets.
Amazon Redshift V2 Connector
This release includes the following enhancements for Amazon Redshift V2 Connector:
Mapping in SQL ELT mode
You can read complex files such as Avro, JSON, ORC, and Parquet from an Amazon S3 source.
You can use the Aggregator, Joiner, Lookup, and Union transformations in a mapping.
You can use an upsert operation to update existing records and insert new records in an Amazon Redshift target.
You can preview data in transformations configured midstream in the mapping flow.
You can push certain aggregate and conditional expression functions to Amazon Redshift.
For the list of functions that you can push down to Amazon Redshift, see the
Amazon Redshift V2 Connector
documentation.
The JDBC driver version is upgraded to 2.1.0.30.
Business 360 FEP Connector
You can now use Business 360 FEP Connector to export the match pairs that are generated by the match process. Additionally, when you export master records, you can also export the identifier details of their source records, such as source primary key and source system, by using the xrefLinks field group. You can also export dynamic field definitions and dynamic field values.
Databricks Connector
This release includes the following enhancements for Databricks Connector:
Mapping in SQL ELT mode
You can use Databricks objects as lookups in a mapping that reads from Amazon S3 or Microsoft Azure Data Lake Storage Gen2.
You can push certain AI and window functions to Databricks.
For the list of functions that you can push down to Databricks, see the
Databricks Connector
documentation.
You can stage data in a Volume in Databricks when you run a Databricks mapping. Volumes in Databricks are Unity Catalog objects used to manage and secure non-tabular datasets such as files and directories.
Datacom CDC Connector
When you restart a task that uses a CDC connector to connect to the source, you can now choose whether to perform a cold start or warm start from the job details page. A cold start restarts the task from the beginning, without recovery. When you cold start a failed task that has recovery enabled, Cloud Data Integration does not process the recovery data. A warm start resumes processing from the point where the task ended, with recovery. Typically, if a task fails after an initial cold start, a warm start is used to restart the task with recovery.
The cold start option is not supported for tasks that have Snowflake targets.
Db2 for i CDC Connector
When you restart a task that uses a CDC connector to connect to the source, you can now choose whether to perform a cold start or warm start from the job details page. A cold start restarts the task from the beginning, without recovery. When you cold start a failed task that has recovery enabled, Cloud Data Integration does not process the recovery data. A warm start resumes processing from the point where the task ended, with recovery. Typically, if a task fails after an initial cold start, a warm start is used to restart the task with recovery.
The cold start option is not supported for tasks that have Snowflake targets.
Db2 for LUW CDC Connector
When you restart a task that uses a CDC connector to connect to the source, you can now choose whether to perform a cold start or warm start from the job details page. A cold start restarts the task from the beginning, without recovery. When you cold start a failed task that has recovery enabled, Cloud Data Integration does not process the recovery data. A warm start resumes processing from the point where the task ended, with recovery. Typically, if a task fails after an initial cold start, a warm start is used to restart the task with recovery.
The cold start option is not supported for tasks that have Snowflake targets.
Db2 for z/OS CDC Connector
When you restart a task that uses a CDC connector to connect to the source, you can now choose whether to perform a cold start or warm start from the job details page. A cold start restarts the task from the beginning, without recovery. When you cold start a failed task that has recovery enabled, Cloud Data Integration does not process the recovery data. A warm start resumes processing from the point where the task ended, with recovery. Typically, if a task fails after an initial cold start, a warm start is used to restart the task with recovery.
The cold start option is not supported for tasks that have Snowflake targets.
Google BigQuery V2
Connector
This release includes the following enhancements for
Google BigQuery V2
Connector:
Mapping in SQL ELT mode
You can push the following functions to
Google BigQuery
in a mapping:
Authenticated Encryption with Associated Data (AEAD) encryption functions
Net functions
Window functions
For the list of functions that you can push to
Google BigQuery
, see the
Google BigQuery Connectors
documentation.
You can use
Google BigQuery
objects as lookups in a mapping that reads from Amazon S3 or Google Cloud Storage objects.
When you query the source object, you can perform the following actions:
Configure the
Is Null
and
Is Not Null
operators in the platform filter to include records where specific fields are either filled or empty.
Sort the input data.
IBM MQ Connector
You can recover unprocessed messages from a failed IBM MQ mapping task and resume the task from the point of failure if the task crashes.
IDMS CDC Connector
When you restart a task that uses a CDC connector to connect to the source, you can now choose whether to perform a cold start or warm start from the job details page. A cold start restarts the task from the beginning, without recovery. When you cold start a failed task that has recovery enabled, Cloud Data Integration does not process the recovery data. A warm start resumes processing from the point where the task ended, with recovery. Typically, if a task fails after an initial cold start, a warm start is used to restart the task with recovery.
The cold start option is not supported for tasks that have Snowflake targets.
IMS CDC Connector
When you restart a task that uses a CDC connector to connect to the source, you can now choose whether to perform a cold start or warm start from the job details page. A cold start restarts the task from the beginning, without recovery. When you cold start a failed task that has recovery enabled, Cloud Data Integration does not process the recovery data. A warm start resumes processing from the point where the task ended, with recovery. Typically, if a task fails after an initial cold start, a warm start is used to restart the task with recovery.
The cold start option is not supported for tasks that have Snowflake targets.
Marketo V3 Connector
When you retrieve leads in bulk from Marketo, you can filter records based on the date they were created or updated.
Microsoft Fabric Data Warehouse Connector
This release includes the following enhancements for Microsoft Fabric Data Warehouse Connector:
You can use key range partitioning to distribute rows of source data based on the partition key and the range of values for each partition.
You can specify an SQL override statement to override the default SQL query that reads data from a Microsoft Fabric Data Warehouse source.
You can configure an update override to override the default update query that the Secure Agent generates for the update operation in a mapping or mapping task.
You can create simple or advanced data filters in a Source transformation to fetch specific data of a particular object.
To look up data from Microsoft Fabric Data Warehouse, you can configure the following Lookup transformations:
Connected and unconnected cached Lookup transformation in a mapping and mapping in advanced mode
Connected uncached Lookup transformation in a mapping
Microsoft Fabric Lakehouse Connector
This release includes the following enhancements for Microsoft Fabric Lakehouse Connector:
You can use key range partitioning to distribute rows of source data based on the partition key and the range of values for each partition.
You can specify an SQL override statement to override the default SQL query that reads data from a Microsoft Fabric Lakehouse source.
You can create simple or advanced data filters in a Source transformation to fetch specific data of a particular object.
To look up data from Microsoft Fabric Lakehouse, you can configure the following Lookup transformations:
Connected and unconnected cached Lookup transformation in a mapping and mapping in advanced mode
Connected uncached Lookup transformation in a mapping
Microsoft SQL Server CDC Connector
When you restart a task that uses a CDC connector to connect to the source, you can now choose whether to perform a cold start or warm start from the job details page. A cold start restarts the task from the beginning, without recovery. When you cold start a failed task that has recovery enabled, Cloud Data Integration does not process the recovery data. A warm start resumes processing from the point where the task ended, with recovery. Typically, if a task fails after an initial cold start, a warm start is used to restart the task with recovery.
The cold start option is not supported for tasks that have Snowflake targets.
Microsoft SQL Server Connector
This release includes the following enhancements for Microsoft SQL Server Connector:
You can connect to SQL Server 2022 version.
You can run Microsoft SQL Server mappings using the upgraded ODBC driver version 08.02.1801. The driver is available in the
When you restart a task that uses a CDC connector to connect to the source, you can now choose whether to perform a cold start or warm start from the job details page. A cold start restarts the task from the beginning, without recovery. When you cold start a failed task that has recovery enabled, Cloud Data Integration does not process the recovery data. A warm start resumes processing from the point where the task ended, with recovery. Typically, if a task fails after an initial cold start, a warm start is used to restart the task with recovery.
The cold start option is not supported for tasks that have Snowflake targets.
ODBC Connector
When you use DB2 subtype in the ODBC connection, you can use an SQL transformation in a mapping to call a stored procedure in a DB2 endpoint.
Oracle CDC V2 Connector
When you restart a task that uses a CDC connector to connect to the source, you can now choose whether to perform a cold start or warm start from the job details page. A cold start restarts the task from the beginning, without recovery. When you cold start a failed task that has recovery enabled, Cloud Data Integration does not process the recovery data. A warm start resumes processing from the point where the task ended, with recovery. Typically, if a task fails after an initial cold start, a warm start is used to restart the task with recovery.
The cold start option is not supported for tasks that have Snowflake targets.
PostgreSQL CDC Connector
When you restart a task that uses a CDC connector to connect to the source, you can now choose whether to perform a cold start or warm start from the job details page. A cold start restarts the task from the beginning, without recovery. When you cold start a failed task that has recovery enabled, Cloud Data Integration does not process the recovery data. A warm start resumes processing from the point where the task ended, with recovery. Typically, if a task fails after an initial cold start, a warm start is used to restart the task with recovery.
The cold start option is not supported for tasks that have Snowflake targets.
SAP HANA CDC Connector
When you restart a task that uses a CDC connector to connect to the source, you can now choose whether to perform a cold start or warm start from the job details page. A cold start restarts the task from the beginning, without recovery. When you cold start a failed task that has recovery enabled, Cloud Data Integration does not process the recovery data. A warm start resumes processing from the point where the task ended, with recovery. Typically, if a task fails after an initial cold start, a warm start is used to restart the task with recovery.
The cold start option is not supported for tasks that have Snowflake targets.
Snowflake Data Cloud Connector
This release includes the following enhancements for Snowflake Data Cloud Connector:
Mapping in SQL ELT mode
You can use Snowflake objects as lookups in a mapping that reads from Amazon S3 or Microsoft Azure Data Lake Storage Gen2.
You can push certain window functions to Snowflake.
For a list of functions that you can use, see the Snowflake Data Cloud Connector documentation.
You can use Microsoft Azure Active Directory as an external OAuth authorization server to authenticate Snowflake.
VSAM CDC Connector
When you restart a task that uses a CDC connector to connect to the source, you can now choose whether to perform a cold start or warm start from the job details page. A cold start restarts the task from the beginning, without recovery. When you cold start a failed task that has recovery enabled, Cloud Data Integration does not process the recovery data. A warm start resumes processing from the point where the task ended, with recovery. Typically, if a task fails after an initial cold start, a warm start is used to restart the task with recovery.
The cold start option is not supported for tasks that have Snowflake targets.
ASK INFAPreview
