Database ingestion incremental load and combined initial and incremental load jobs can access the Oracle redo logs for CDC processing in alternative ways, depending on your environment and requirements.
Direct log access
Database ingestion jobs can directly access the physical Oracle redo logs on the on-premises source system to read change data.
If you store the logs on solid-state disk (SSD), this method can provide the best performance.
The following image shows the data flow:
The Oracle database writes change records to the database log files on disk.
The Mass Ingestion Databases CDC Reader reads the physical log files and extracts change records from the log files for the source tables of CDC interest.
The Mass Ingestion Databases CDC Writer reads the change records.
The CDC Writer applies the change records to the target.
NFS-mounted logs
Database ingestion jobs can access to Oracle database logs from shared disk by using a Network File Sharing (NFS) mount or another method such as Network Attached Storage (NAS) or clustered storage.
The following image shows the data flow:
The Oracle database writes change records to database log files. The log files are written to shared disk.
The shared disk can be on any system that allows the files to appear as local to both the database and Secure Agent hosts. This sharing can be achieved by using NFS, as shown above, or by using Network Attached Storage (NAS) or clustered storage.
The Mass Ingestion Databases CDC Reader reads the log files from the NFS server over the network and extracts the change records for the source tables of CDC interest.
The Mass Ingestion Databases CDC Writer reads the change records.
The CDC Writer applies the change records to the target.
ASM-managed logs
Database ingestion jobs can access Oracle redo logs that are stored in an Oracle Automatic Storage Management (ASM) system. To read change data from the ASM-managed redo logs, the ASM user must have SYSASM or SYSDBA authority on the ASM instance.
The following image shows the data flow:
The Oracle database writes change records to the ASM-managed database log files.
The Mass Ingestion Databases CDC Reader reads the ASM-managed log files and extracts the change records for the source tables of CDC interest.
The Mass Ingestion Databases CDC Writer reads the change records.
The CDC Writer applies the change records to the target.
ASM-managed logs with a staging directory
Database ingestion jobs can access ASM-managed redo logs from a staging directory in the ASM environment. In comparison to using ASM only, this method can provide faster access to the log files and reduce I/O on the ASM system. To read change data from the ASM-managed logs, the ASM user must have SYSASM or SYSDBA authority on the ASM instance.
The following image shows the data flow:
The Oracle database writes change records to the ASM-managed log files.
ASM copies the logs to a staging directory.
The staging directory must be on shared disk, such as an NFS mount, so that ASM can write data to it and database ingestion jobs can read data from it.
The Mass Ingestion Databases CDC Reader reads the log files in the staging directory and extracts the change records for the source tables of CDC interest.
The Mass Ingestion Databases CDC Writer reads the change records.
The CDC Writer applies the change records to the target.
BFILE access to logs in the Oracle server file system by using directory objects
On an on-premises Oracle source system, you can configure Mass Ingestion Databases to read online and archived redo logs from the local Oracle server file system by using Oracle directory objects with BFILE locators. You must create Oracle directory objects named ARCHIVELOG_DIR and ONLINELOG_DIR that point to the locations of the Oracle redo log files. For information about configuring BFILE access, see
Configuring BFILE access to Oracle redo logs in the Oracle file system.
The following image shows the data flow:
The Oracle database writes change records to the redo log files in the local Oracle server file system. When a database ingestion task needs to read log files, it connects to Oracle and issues a select request that references the ARCHIVELOG_DIR or ONLINELOG_DIR directory object to access the logs.
The Mass Ingestion Databases CDC Reader reads the log file and extracts the change records for the source tables of CDC interest.
The Mass Ingestion Databases CDC Writer reads the change records.
The CDC Writer applies the change records to the target.
ASK INFAPreview
