Databricks Delta Connector

Back Next

Configuring pushdown optimization

To optimize a mapping, add the mapping to a task, and then configure pushdown optimization in the

mapping

task.

Create a

mapping

task.

In the

Pushdown Optimization

section on the

Schedule

tab, set the pushdown optimization value to

Full

To Source

If full pushdown optimization is not available, select how Data Integration handles pushdown optimization in the

Pushdown Optimization Fallback Option

menu:

Partial PDO. Default. Data Integration pushes as much transformation logic as possible to the source and target database. The task processes any transformation logic that it can't push to a database. You can use Partial PDO only when you read from and write to Databrick Delta.

Non PDO. The task runs without pushdown optimization.

Fail Task. Data Integration fails the task.

The fallback options are not applicable to mappings in advanced mode.

When you run the mapping task, the transformation logic is pushed to the Databricks Delta database.

Databricks Delta pushdown optimization (SQL ELT)

Download Guide

Watch

Comments