Hi, I'm Ask INFA!
What would you like to know?
ASK INFAPreview
Please to access Bolo.

Table of Contents

Search

  1. Introduction to Databricks Delta Connector
  2. Connections for Databricks Delta
  3. Mappings and mapping tasks with Databricks Delta connector
  4. Databricks Delta pushdown optimization (SQL ELT)
  5. Data type reference

Databricks Delta Connector

Databricks Delta Connector

Databricks compute resources

Databricks compute resources

When you configure a mapping that reads from or writes to Databricks Delta, the Secure Agent, by default, connects to the Databricks SQL warehouse to run the SQL query and process the mapping.
However, you can connect to the Databricks cluster instead of the SQL warehouse to process the mapping. To do so, enable the Secure Agent properties for the design time and runtime. The Secure Agent connects to the analytics cluster to import the metadata at design time and to the job cluster to run the mappings.
The Secure Agent runs a Spark job in the cluster to read data from or write data to Databricks Delta tables.
You can only use the SQL warehouse to configure mappings in the Windows operating system.

0 COMMENTS

We’d like to hear from you!