Hi, I'm Ask INFA!
What would you like to know?
ASK INFAPreview
Please to access Ask INFA.

Table of Contents

Search

  1. Introducing Mass Ingestion
  2. Getting Started with Mass Ingestion
  3. Connectors and Connections
  4. Mass Ingestion Applications
  5. Mass Ingestion Databases
  6. Mass Ingestion Files
  7. Mass Ingestion Streaming
  8. Monitoring Mass Ingestion Jobs
  9. Asset Management
  10. Troubleshooting

Mass Ingestion

Mass Ingestion

Guidelines for Google BigQuery targets

Guidelines for Google BigQuery targets

Consider the following guidelines when you use Google BigQuery targets:
  • When you use a Google BigQuery target for the first time, perform the following steps before you configure an
    application ingestion
    task for the target:
    1. Download the Google BigQuery JDBC driver from the Google Cloud website.
    2. Copy the JDBC driver jar files to the following directory:
      <Secure_Agent_installation_directory>/apps/Database_Ingestion/ext/
    3. Restart the Secure Agent.
  • You must have a service account in your Google account to access Google BigQuery and Google Cloud Storage.
  • Ensure that you have the client_email, project_id, private_key, private_key_id, client_id, and region ID values for the service account. You must enter these details when you create a Google BigQuery connection.
    Specify the private_key_id and client_id values in the
    Provide Optional Properties
    field of the connection properties. Use the following format:
    "private_key_id": "<
    private_key_id_value
    >", "client_id": "<
    client_id_value
    >"
  • If you want to configure a timeout interval for a Google BigQuery connection, specify the timeout interval property in the
    Provide Optional Properties
    field of the connection properties. Use the following format:
    "timeout": "<
    timeout_interval_in_seconds
    >"
  • You must have read and write access to the following entities:
    • Google BigQuery datasets that contain the target tables.
    • Google Cloud Storage path where
      Mass Ingestion Applications
      creates the staging file.
  • Application ingestion jobs configured for Google BigQuery targets do not replicate the modification and renaming of source fields on the target.
  • You must have the following permissions to write data to a Google BigQuery table:
    • bigquery.datasets.get
    • bigquery.datasets.getIamPolicy
    • bigquery.models.*
    • bigquery.routines.*
    • bigquery.tables.create
    • bigquery.tables.delete
    • bigquery.tables.export
    • bigquery.tables.get
    • bigquery.tables.getData
    • bigquery.tables.list
    • bigquery.tables.update
    • bigquery.tables.updateData
    • bigquery.tables.updateTag
    • resourcemanager.projects.get
    • resourcemanager.projects.list
    • bigquery.jobs.create

0 COMMENTS

We’d like to hear from you!