Hi, I'm Ask INFA!
What would you like to know?
ASK INFAPreview
Please to access Bolo.

Table of Contents

Search

  1. Introduction to Databricks Delta Connector
  2. Connections for Databricks Delta
  3. Mappings and mapping tasks with Databricks Delta connector
  4. Databricks Delta pushdown optimization (SQL ELT)
  5. Data type reference

Databricks Delta Connector

Databricks Delta Connector

Configure Secure Agent properties

Configure Secure Agent properties

When you configure mappings, the SQL warehouse processes the mapping by default.
To process the mappings on Databricks cluster, enable the Secure Agent properties.
To connect to analytics cluster and job cluster, enable the Secure Agent properties for design time and runtime respectively.

Setting the property for design time processing

Before you can import metadata and design mappings or mappings in advanced mode, perform the following steps:
  1. In
    Administrator
    , select the Secure Agent listed on the
    Runtime Environments
    tab.
  2. Click
    Edit
    .
  3. In the
    System Configuration Details
    section, select Data Integration Server as the
    Service
    and Tomcat JRE as the
    Type
    .
  4. Edit the
    JRE_OPTS
    field and set the value to
    -DUseDatabricksSql=false
    .
This image displays the JRE_OPTS property for the Tomcat JRE type.

Setting the property for runtime processing

  1. In
    Administrator
    , select the Secure Agent listed on the
    Runtime Environments
    tab.
  2. Click
    Edit
    .
  3. In the
    System Configuration Details
    section, select Data Integration Server as the
    Service
    and DTM as the
    Type
    .
  4. Edit the
    JVMOption
    field.
    1. To run mappings, set the value to
      -DUseDatabricksSql=false
      . This image shows the JVMOption property for mappings.
    2. To run mappings enabled with pushdown optimization, set the value to
      -DUseDatabricksSqlForPdo=false
      . This image shows the JVMOption property for mappings enabled with pushdown optimization.

0 COMMENTS

We’d like to hear from you!