Architecture
The platform and cluster nodes each independently need access to the Ionic service. This is because no sensitive information is transferred to the cluster during job submission. The only information relating to Ionic, that is submitted, is the location of Ionic profiles, and the name of the environment variable storing the password.
Implementation flow
 |
When the platform encounters a situation whereby secret key material is required, it checks what is configured in the Job’s Policy or Environment to find the name of the key to be used.
To maintain consistency, a PDD will always use the same version of the key.
The platform creates a Spark job definition and submits this to the cluster. No sensitive information is sent as part of the job submission process. The relevant information sent as part of this request is:
The path of the Ionic Profile Persistor file, which must be common across all nodes
The name of the environment variable containing the Persistor’s password.
The name and ID of the correct version of each key to be used in the job.
In the case of JDBC credentials, the job definition only contains the key name (and not the ID of a particular version) of the key to be used. Both the platform node and the Spark processing nodes will request the latest version of the key from the KMS.
When a key is required by the Job, the spark processing nodes use the same authentication process as in step 2 using their own local Ionic Profile Persistor and
IONIC_PASSWORDenvironment variable to access the KMS.
Note
The Privitar Policy Manager application reads the IONIC_PASSWORD environment variable containing the password for the Ionic Profile at startup time only. It does not require continued access to the environment variable after startup and is available in memory within the application. Whilst the application is running, the environment variable is no longer required to be set until the application undergoes a restart.