Hi, I'm Ask INFA!
What would you like to know?
ASK INFAPreview
Please to access Ask INFA.

Table of Contents

Search

  1. Introducing Mass Ingestion
  2. Getting Started with Mass Ingestion
  3. Connectors and Connections
  4. Mass Ingestion Applications
  5. Mass Ingestion Databases
  6. Mass Ingestion Files
  7. Mass Ingestion Streaming
  8. Monitoring Mass Ingestion Jobs
  9. Asset Management
  10. Troubleshooting

Mass Ingestion

Mass Ingestion

External stage

External stage

When you configure a file ingestion task to load files from a Microsoft Azure Blob Storage or Amazon S3 source to the Snowflake Data Cloud tables, specify the external staging directory to use in Snowflake.
You must specify the external stage name for the specific connection type that you want to use in the
Target Options
section in the file ingestion task.
The external stage field value is mandatory when you run a file ingestion task to load files from Microsoft Azure Blob Storage to Snowflake Data Cloud where the connection type in the source is Microsoft Azure Blob Storage V2. When the source connection type is Amazon S3 V2, and you do not specify an external stage for Amazon S3 V2 in the Snowflake Data Cloud target options, Snowflake creates an external stage directory by default.
Ensure that the source directory path in the
Source Options
of the file ingestion task is the same as the directory path provided in the URL of the external stage created for the Microsoft Azure Blob Storage V2 or Amazon S3 V2 connection in Snowflake Data Cloud.
For example, an external stage for Microsoft Azure Blob Storage created using an Azure account name and a blob container with a folder path has the following stage URL:
'azure://<URL>/<blob container>/<folder path>'
. The stage uses the file format you specify in the
Target Options
of the file ingestion task.
The following image shows the stage name and the stage URL for a Microsoft Azure Blob Storage V2 connection in Snowflake Data Cloud:
You can view the details of the stage URL created in Snowflake.
In the example, the stage URL is
azure://adapterdevblob.blob.core.windows.net/snowflakemi/MI/
and the external stage name is
MFT_BLOB1
.
When you create a file ingestion job, in the
Folder Path
field in the
Source Options
of the Microsoft Azure Blob Storage V2 source, specify the following
<Blob Container>/<folder path>
path from the stage URL:
/snowflakemi/MI
The following image shows the specified source folder path in the
Source Options
section:
You can view the configured details on the Microsoft Azure Blob Storage Source tab in a mass ingestion task.
In the
Target Options
for Snowflake Data Cloud, specify the following name of the created external stage:
MFT_BLOB1
The following figure shows the configured external stage field in the
Target Options
section:
Snoflake Data Cloud target properties

0 COMMENTS

We’d like to hear from you!