The connection type populates automatically based on the connection that you select.

The operation type to write data to ADLS Gen2 file.

Default is

Append

.

Append
. Add data to an existing file inside a directory.
Overwrite
. Delete existing data in an existing file and insert newly read data.
Fail
. Write data to an existing file fails.
Rollover
. Close the current file to which data is being written to and create a new file based on the configured rollover value.
When you edit or redeploy a
streaming ingestion
job that contains a target with the rollover strategy, all the files in the staging directory are moved to the target directory even if the defined rollover conditions are not met.

Path to the staging directory in ADLS Gen2.

Specify the staging directory where you want to stage the files when you write data to ADLS Gen2. Ensure that the directory has sufficient space and you have write permissions to the directory.

Applicable when you select the

Write Strategy

as

Rollover

.

While configuring an ADLS Gen 2 target in a

streaming ingestion

job, if you do not specify any value for the rollover properties, the files remain in the interim directory. When you stop or undeploy the

streaming ingestion

job, these files in the interim directory are moved to the target location, by default.

Target file size, in kilobytes (KB), at which to trigger rollover.

Applicable when you select the

Write Strategy

as

Rollover

.

Number of events or messages that you want to accumulate for the rollover.

Applicable when you select the

Write Strategy

as

Rollover

.

Length of time, in milliseconds, for a target file to roll over. After the time period has elapsed, the target file rolls over.

Applicable when you select the

Write Strategy

as

Rollover

.

Use the

$expression$

format for the regular expression.

$expression$

evaluates the data and sends the matching data to capturing group 1.

The ADLS Gen2 directory that you use to write data.

Default is root directory.

The directory path specified while creating the target overrides the path specified while creating a connection.

Optional. Compression format to use before the

streaming ingestion

task writes data to the target file.

Use one of the following formats:

None
Gzip
Bzip2
Zlib
Deflate

Default is

None

.

To read a compressed file from the data lake storage, the compressed file must have specific extensions. If the extensions used to read the compressed file are not valid, the Secure Agent does not process the file.