Communities
A collaborative platform to connect and grow with like-minded Informaticans across the globe
Product Communities
Connect and collaborate with Informatica experts and champions
Discussions
Have a question? Start a Discussion and get immediate answers you are looking for
User Groups
Customer-organized groups that meet online and in-person. Join today to network, share ideas, and get tips on how to get the most out of Informatica
Get Started
Community Guidelines
Knowledge Center
Troubleshooting documents, product guides, how to videos, best practices, and more
Knowledge Base
One-stop self-service portal for solutions, FAQs, Whitepapers, How Tos, Videos, and more
Support TV
Video channel for step-by-step instructions to use our products, best practices, troubleshooting tips, and much more
Documentation
Information library of the latest product documents
Velocity (Best Practices)
Best practices and use cases from the Implementation team
Learn
Rich resources to help you leverage full capabilities of our products
Trainings
Role-based training programs for the best ROI
Certifications
Get certified on Informatica products. Free, Foundation, or Professional
Product Learning Paths
Free and unlimited modules based on your expertise level and journey
Resources
Library of content to help you leverage the best of Informatica products
Tech Tuesdays Webinars
Most popular webinars on product architecture, best practices, and more
Product Availability Matrix
Product Availability Matrix statements of Informatica products
SupportFlash
Monthly support newsletter
Support Documents
Informatica Support Guide and Statements, Quick Start Guides, and Cloud Product Description Schedule
Product Lifecycle
End of Life statements of Informatica products
Ideas
Events
Change Request Tracking
Marketplace
English
  • Data Engineering Integration H2L
  • All Products

Table of Contents

Search

  1. Abstract
  2. Supported Versions
  3. Integrating Informatica® Big Data Management 10.2.2 HF1 SP1 with Qubole

Integrating Informatica® Big Data Management 10.2.2 HF1 SP1 with Qubole

Integrating Informatica® Big Data Management 10.2.2 HF1 SP1 with Qubole

Back Next

Configure the Cluster

Configure the Cluster

In the next panel of the cluster creation wizard, you configure cluster settings.
  1. Configure cluster settings.
    The following table describes the settings to configure:
    Property
    Description
    Cluster Labels
    Specify a string to use as a prefix to cluster node host names.
    Spark Version
    Select
    2.2 latest -- Spark 2.2.1
    .
    You must perform separate configuration steps to enable Spark version 2.2.1. See the following topics in this article:
    Notebook Interpreter Node
    Qubole protocol for using Qubole notebooks. Accept the default setting.
    Big Data Management version 10.2x does not support integration with Qubole notebooks.
    Master Node Type
    Select an EC2 instance type from the drop down list.
    For the master node, Informatica requires a minimum 16 cores and 64 GB memory.
    Worker Node Type
    Select an EC2 instance type from the drop down list.
    For the worker node, Informatica requires a minimum 16 cores and 64 GB memory.
    Use Multiple Worker Node Types
    Select this option if you want to specify multiple EC2 instance types for worker nodes.
    Minimum Worker Nodes
    Specify the minimum number of worker nodes to provision.
    Maximum Worker Nodes
    Specify the maximum number of worker nodes to provision.
    Region
    Specify the AWS region in which to create the cluster.
    Availability Zone
    Availability zone where the cluster is deployed. Accept the default value.
    Default: Any.
    EBS Volume Count
    Count of EBS (elastic block storage) volumes to be mounted to an instance as reserved disks.
    Default: 0
    EBS Volume Type
    Type of EBS volume to be mounted to an instance as reserved disk. Choose from the following options:
    • ssd (gp2 SSD). General purpose solid state drive.
    • standard (standard HDD). Standard-sized hard disk drive.
    • st1 (HDD). Throughput-optimized hard disk drive.
    • sc1 (HDD). Cold hard disk drive.
    For more information about EBS volume types, see Amazon documentation.
    EBS Volume Size
    Size (in GB) of each EBS volume to be mounted to an instance as reserved disk.
    Enable EBS Upscaling
    Enable dynamic block storage up-scaling.
    Default is disabled.
    Node Bootstrap File
    Name of the file that contains bootstrap instructions to run after the Qubole cluster is provisioned.
    The file must be in the default path displayed above the text entry pane.
    For more information about node bootstrap files, see Qubole documentation.
    Disable Automatic Cluster Termination
    Check this option to disable Qubole automatic cluster termination. Informatica requires disabling of automatic termination.
    Idle Cluster Timeout
    Applies only to automatic cluster termination. Disabled if automatic cluster termination is disabled.
  2. Click
    Next
     to continue to the next panel.
    In the next panel of the cluster creation wizard, you can configure cluster composition. It is not necessary to change any of the default settings in this panel.
  3. Click
    Next
     to continue to the next panel, where you configure advanced properties.
0 COMMENTS
We’d like to hear from you!