Databricks Delta Connector

Back Next

Override the update operation

You can specify an update override to override the update query that the Secure Agent generates for the update operation.

When you configure an update override, the Secure Agent uses the query that you specify, stages the data in files, and then loads that data into a temporary table using the merge command. The data from the temporary table is then loaded to the Databricks target table. The syntax that you specify for the update query must be supported by Databricks Delta:

Specify the update override query in the following format:

MERGE INTO <Target table name here> AS A USING :TU AS B ON A.<Column1> = B.<Column1> WHEN MATCHED THEN UPDATE SET A.<Column2> = B.<Column2>, A.<Column3>= B.<Column3> ... A.<ColumnN>= B.<ColumnN>

where, :TU
represents the incoming data source for the target port. The Secure Agent replaces
:TU
with a temporary table name while running the mapping and does not validate the update query.

When you configure an update override in a mapping to write to Databricks Delta, consider the following rules and guidelines:

Ensure that the column names for :TU matches the target table column names.

Specify the update query with a valid SQL syntax because Databricks Delta Connector replaces :TU with a temporary table name and does not validate the update query.

Do not change the order of the column in the mappings when you configure the update override option.

The update query in the mapping must not contain unconnected fields to the target.

You can only override a single SQL query at runtime.

Ensure that the number of rows processed using an SQL query is the same as the number of rows in the target transformation.

You cannot perform an update override on Insert, Upsert, and Delete target operations.

Rename Saved Search

Table of Contents

Databricks Delta Connector

Databricks Delta Connector

Override the update operation

Override the update operation