You must configure a Databricks Delta connection with simple or hybrid mode when you enable pushdown optimization in a mapping task.
Source properties
When you configure pushdown optimization, the mappings support the following advance properties for a Databricks Delta source:
Source Object Type
Single
Multiple
Query
Parameter
When you use the query source type to read from Databricks Delta, you can choose to retain the field metadata and save the mapping. Even if you edit the query and run the mapping, the field metadata specified at design time is retained.
Query Options
Filter. You can use both simple and advanced filter conditions.
Database Name
Table Name
Contains, Ends With, and Starts With filter operators are not applicable when you use source filter to filter records.
Target properties
When you configure pushdown optimization, the mappings support the following properties for an Databricks Delta target:
Target Object Type
Single
Parameter
Create New at Runtime
Operation
Insert
Update
Upsert
Delete
Create Target
Target Database Name
Target Table Name
Update Mode
Write Disposition for Insert operation.
You cannot run pre-SQL or post-SQL commands in the source and target when you configure mappings for full pushdown optimization.
Lookup properties
When you configure pushdown optimization, the mappings support the following advance properties for a Databricks Delta lookup:
Source Object Type
Single
Query
Parameter
Multiple Matches for cached lookups
Un-cached unconnected lookups are not supported for pushdown optimizations. For cached lookups, only
return all rows
is supported.
Database Name
Table Name
If you configure advanced properties that are not supported, the Secure Agent either ignores the properties or logs a pushdown optimization validation error in the session logs file. If the Secure Agent logs an error in the session log, the mappings run in the Informatica runtime environment without full pushdown.
ASK INFAPreview
