In the read and write operations for a complex file data object, you cannot edit the precision and scale of elements within a field that is of a complex data type.

For example, if Field1 is of type array with string elements, you cannot edit the precision and scale of the string elements.

Workaround: Create a separate Sqoop connection for each Sqoop data source with the

-split-by

option, and run a column profile on the data source.

Workaround: Use the

hdp-connector-for-teradata-1.4.1.2.3.2.0-2950-distro.tar.gz

JAR.

Mappings that read data from a Teradata source and contain the != (not equal) operator in the filter override query fail. This issue occurs if you run the Teradata Parallel Transporter mapping on a Hortonworks cluster and on the Blaze engine.

Workaround: Use a native expression with the ne operator instead of the != operator.

Workaround: Before you run the mapping, create a table in the target database with columns corresponding to the connected ports.

Workaround: On the Sqoop data node, run the following command to kill the Sqoop map-reduce jobs:

yarn application -kill <application_ID>

A column profile runs indefinitely when the following conditions are true:

The Hive data source for the profile resides on an Azure HDInsight cluster that uses WASB storage.
You create a column profile on the Hive source and enable data domain discovery.
You run the profile on the Hive engine in the Hadoop environment.
You do not use the JDBC connection as the profiling warehouse connection.

When the Data Integration Service is configured to run with operating system profiles and you push the mapping to an HDInsight cluster with ADLS as storage, the mapping fails with the following error:

When you choose

Return All

for an unconnected lookup, mapping validation fails with the following unrelated error:

Workaround: If possible, increase partitioning on the source.

When the Blaze engine runs a mapping that uses source or target files in the WASB location on a cluster, the mapping fails with an error like:

 java.lang.RuntimeException:
[<error_code>] The Integration Service failed to run Hive
query [exec0_query_6] for task [exec0] due to following error:
<error_code> message [FAILED: ... Cannot run program
"/usr/lib/python2.7/dist-packages/hdinsight_common/decrypt.sh":
error=2, No such file or directory], ...

The mapping fails because the cluster attempts to decrypt the data but cannot find a file needed to perform the decryption operation.

Workaround: Find the following files on the cluster and copy them to the

/usr/lib/python2.7/dist-packages/hdinsight_common

directory on the machine that runs the Data Integration Service:

key_decryption_cert.prv
decrypt.sh

Workaround: To run a map-reduce job in a particular yarn queue, configure the following property in the

Sqoop Arguments

field of the JDBC connection:

-Dmapreduce.job.queuename=<NameOfTheQueue>

To run a Spark job in a particular yarn queue, configure the following property in the Hadoop connection:

spark.yarn.queue=<NameOfTheQueue>

When you create or import type definition libraries using JSON objects with key names that have the dot operator (.), the following validation error occurs:

Evaluation failed and was not completed. Check the Developer tool logs for details.

A mapping that runs on the Hive engine fails when it meets the following conditions:

The mapping writes to multiple targets, and at least one is a Hive target.
The mapping contains a Router transformation.
Storage-based authorization is enabled on the mapping

New mapping jobs fail when the Data Integration Service machine reaches the maximum configured memory.

Memory leaks occur on the Data Integration Service machine when the Spark engine heap memory reaches 4 GB.

Workaround: To convert relational data to a hierarchical data of more than four levels, develop more than one mapping to stage the intermediate data. For example, develop a mapping that converts relational data to a hierarchical data up to three levels. Use the hierarchical data in another mapping to generate a hierarchical data of four levels.

Mappings fail when you use the impersonation user to read or write files within an HDFS encryption zone in a Hadoop cluster and operating system profiles are enabled for the Data Integration Service.

Workaround: Read and write the files outside of the encryption zone.