Skip to main content

User Guide

About the Event Data

This section provides an overview of the Event data that is captured, including the type of data that is collected, how the data is stored to ensure there are no privacy or security concerns and the format of the stored data.

Data Capture Process

There is a two-step event data capture process:

  1. Gather instrumentation data from Privitar into Event Log files, using the Event Broker.

  2. Use the Uploader to upload the files to a Privitar-provided, over secure communication to a secure data store such as an Amazon AWS S3 bucket.

These steps are performed separately and asynchronously. To ensure that no data leaves your environment without your consent, the Uploader is configured to be triggered by you from your environment.

Data categories

The data is captured in the form of events that are triggered by a certain defined event taking place on the platform.

The data capture process captures both historic event data as well as current event data.

Historic data

Event level data for the following instrumentation categories are extracted from pre-existing logs in your current configuration database:

  • Schema

  • Policies

  • PDDs

  • Rules

  • Jobs

  • Batch Job runs

  • Team

  • Users

  • Environments

These intermediary files will be generated into synthetic events that can be written to the Event Log file.

All objects like Schemas, Rules, Policies and PDDs will only be identifiable by random IDs. When the events are captured, information that can be considered business sensitive, such as names or any personal identification data will be stripped from the event before it is added to the Event Log.

It is possible to confirm that this de-identification of the data has occurred by downloading the data and reviewing all the generated files in a Text Editor.

Current data

Privitar services such as Policy manager generate event-level data that is sent to the Event Broker. The following categories of event can be generated:

  • Schema: created, updated and deleted

  • Policy: created, updated and deleted

  • PDD: created, updated, deleted and closed

  • Rules: created, updated and deleted

  • Jobs: created, updated, deleted, started

  • Job_Run: Batch and Data Flow

  • PPO: number of PPOs

  • Teams: created, updated and deleted

  • Team membership: added and removed

  • Token mapping removal: start and finish

  • Users: created, updated, logon, type of user

  • Environments: created, updated and deleted

  • Data Flow Processors: created, closed and metrics

  • Policy Manager: started

  • Service Status: current status and changes to the current status

For each event that is generated, the following details are always included:

  • event_type; type of event generated.

  • timestamp; time and date that the event was created.

  • unique_id; unique identifier for the event.

  • event_source; indicates if the event is taken from the Config database or directly from a service.

There will also be additional details captured that are specific to the event.

For example, this is the format of the event (PDD_CREATED) that is generated when a PDD is created:

"@timestamp": "2020-06-23T08:37:05.801Z",
    "eventSource": "SYNTHESISED",
    "eventType": "PDD_CREATED",
    "parameters": {
      "watermarked": "false",
      "team": {
        "id": "66586",
        "uniqueId": "qk7oshxo"
      },
      "uniqueId": "hzkmk54j",
      "metadata": [
        "Recipient",
        "Description",
        "Approver",
        "Intended Use",
        "Restrictions"
      ],
      "objectType": "com.privitar.agrotera.dashboard.domain.ProtectedDataDomain",
      "environment": {
        "id": "66609",
        "uniqueId": "yqblk6y4"
      },
      "id": "66613"
    },
    "timestamp": "2020-06-23T08:37:05.801",
    "uuid": "f2b5848c-b85b-4d2a-8ad7-d73eb1004ec4",
    "@version": "1",
    "customer": "test-upload"

For more information about the details that are captured for each event, refer to Event Types.

In this section: