Table of Contents

Search

  1. Introduction to Databricks Delta Connector
  2. Connections for Databricks Delta
  3. Mappings and mapping tasks with Databricks Delta connector
  4. Databricks Delta pushdown optimization (SQL ELT)
  5. Data type reference

Databricks Delta Connector

Databricks Delta Connector

Read from Amazon S3 and write to Databricks Delta

Read from Amazon S3 and write to Databricks Delta

You can configure pushdown optimization for a mapping that uses an Amazon S3 V2 connection in the Source transformation to read from Amazon S3 and a Databricks Delta connection in the Target transformation to write to Databricks Delta.

Example

You work for a healthcare organization. Your organization offers a suite of services to manage electronic medical records, patient engagement, telephonic health services, and care coordination services. The organization uses infrastructure based on Amazon Web Services and stores its data on Amazon S3. The management plans to load data to a data warehouse to perform healthcare analytics and create data points to improve operational efficiency. To load data from an Amazon S3 based storage object to Databricks Delta, you must use ETL and ELT with the required transformations that support the data warehouse model.
Use an Amazon S3 V2 connection to read data from a file object in an Amazon S3 source and a Databricks Delta connection to write to a Databricks Delta target. Configure full pushdown optimization in the mapping to optimize the performance.

0 COMMENTS

We’d like to hear from you!
Hi, I'm Bolo!
What would you like to know?
Please to access Bolo.