Each transformation performs a specific function. For example, a Filter transformation filters data from the ingested data based on a specified condition.

When you create a

streaming ingestion

task, adding a transformation is optional. Each transformation type has a unique set of options that you can configure.

You can use the following transformations in

streaming ingestion

tasks:

Combiner
Filter
Python
Splitter
Format Converter

You can add multiple transformations to a

streaming ingestion

task. In such a case, the order of transformations is important because the source data undergoes each transformation in the given order. The output of one transformation becomes the input to the next one in the task flow.

In a

streaming ingestion

task, you can add only one Combiner transformation and one Format Converter transformation. The Format Converter transformation must be the last transformation in the task flow. If the task includes both a Combiner transformation and a Format Converter transformation, the Format Converter transformation must be the last transformation in the task flow, preceded by the Combiner transformation.