Log Queries and Examples
The platform generates platform logs from the proxy. You need the proxy URL to set up your integrated development environment (IDE) when running a query on the proxy. You can get this from within the platform:
Click View Exchange to open the data exchange.
Open the Data Plane tab.
Open the Data Proxy tab to see the proxy host details.
Next, run a query from an IDE, such as VisualStudio or IntelliJ IDEA to produce a log. The proxy produces a two-audit event record; one of action type user.authentication and another of action type policy.resolution.
Note that the query runs against database metadata, not the platform metadata.
For the query audit event, you can set it to be verbose or non-verbose with an application property of: privitar.jdbcproxy.queryengine.verboseAuditLog. By default it is false. If you set it to true, the query audit events will also include information about the request query and the transformations applied to the field.
The audit event does not include information about the execution nor status of the request (query).
Audit Log Field Descriptions
Field | Description |
|---|---|
| A unique audit ID, generated for each event. |
| A system event which generates this audit record. The format of this field is |
| Event type generating this event. For example, |
| The time that the event generated the audit record. |
| The username, group name, or system name of the entity that triggered the event. For certain events, this can be NULL. |
| Whether initiator is a user or system_user. |
| The tenant ID where the event takes place. For certain events, this can be NULL. |
| The system observing the event. |
| Outcome of event is either |
| Optional error message when the outcome is |
| The target system ID observing the event. When running on Kubernetes this is the node ID. In other environments this is the ID of the data plane. |
| The target system type. |
| The impact of an event on the system.
|
| Optional. When running in Kubernetes, this is the namespace and pod where the event takes place. |
| The ID to link to the incoming request ID that generates this event. For workflow events, the |
| Optional. This is the response data that the event generated. |
| Optional. This is the request data for the event. |
Example of the User Info Audit
{
"header": {
"id": "00f5fc6b-a1d2-4db1-be44-9d7360a25176",
"event": {
"eventTime": "2022-11-04T12:36:49.439Z",
"eventType": "ACTIVITY",
"action": "user.authentication"
},
"initiator": {
"id": "29bd6d2a-10a3-44aa-8de7-1f264e22e21c",
"typeURI": "User",
"tenant": "fff7a844-f732-4a65-8280-912eae8eb8d0"
},
"observer": {
"id": "target"
},
"status": {
"outcome": "SUCCESS"
},
"target": {
"id": "01234567-89ab-cdef-0123-456789abcdef",
"typeURI": "service/compute"
},
"severity": "WARNING"
},
"attachments": [
{
"source": {
"namespace": "dpp-proxy",
"pod": "dynamic-proxy-b48ad3d7-7b7c-431f-ad5d-633e5641c0cd-59c677blq52m"
},
"correlationId": "93429272-b9e7-4452-b799-dfeddf5e3a47",
"requestData": {
"context": {
"project": {
"id": "5a078d49-1bc3-4a6f-9caa-1d622f18e7cc"
},
"requestAttributes": {
"request.project": "5a078d49-1bc3-4a6f-9caa-1d622f18e7cc"
}
}
}
}
]
}You can see that "action" tells us what type of audit log it is.
Example of the Non-verbose Query Audit Event
{
"header": {
"id": "40012d8a-f002-4fcf-87cd-549e66d57634",
"event": {
"eventTime": "2022-11-07T06:46:13.024082Z",
"eventType": "ACTIVITY",
"action": "policy.resolution"
},
"initiator": {
"id": "ba992d9f-2c0c-476e-9584-9e8ef923a5bf",
"typeURI": "User",
"tenant": "bc5bbfa3-e07c-4688-8644-a516afe40c1b"
},
"observer": {
"id": "target"
},
"status": {
"outcome": "SUCCESS",
"reason": null
},
"target": {
"id": "ip-10-18-122-83.eu-west-1.compute.internal",
"typeURI": "service/compute"
},
"severity": "WARNING"
},
"attachments": [
{
"source": {
"namespace": "dpp-8193-common-audit-dp",
"pod": "dynamic-proxy-b48ad3d7-7b7c-431f-ad5d-633e5641c0cd-59c677blq52m"
},
"correlationId": "b93069e5-676b-41ce-9c2d-c69c2b5d9360",
"responseData": null,
"requestData": {
"transformations": [
{
"field": [
{
"catalog": "acmedb",
"schema": "library",
"table": "authors",
"field": "firstname"
}
]
},
{
"field": [
{
"catalog": "acmedb",
"schema": "library",
"table": "authors",
"field": "lastname"
}
]
},
{
"field": [
{
"catalog": "acmedb",
"schema": "library",
"table": "authors",
"field": "id"
}
]
}
],
"filters": [
{
"field": {
"catalog": "acmedb",
"schema": "library",
"table": "authors",
"field": "firstname"
},
"expression": "firstname <> 'AAAAA'"
}
]
}
}
]
}Example of the Verbose Query Audit Event for a SQL Query
Turn auditing on, is true by default: privitar.audit.enabled=true
Configure auditing to emit records to Fluent Bit: privitar.audit.auditServiceUrl=true
{
"header": {
"id": "00f5fc6b-a1d2-4db1-be44-9d7360a25176",
"event": {
"eventTime": "2022-11-04T12:36:49.439Z",
"eventType": "ACTIVITY",
"action": "policy.resolution"
},
"initiator": {
"id": "successfulUser",
"typeURI": "User",
"tenant": "fff7a844-f732-4a65-8280-912eae8eb8d0"
},
"observer": {
"id": "target"
},
"status": {
"outcome": "SUCCESS",
"reason": null
},
"target": {
"id": "01234567-89ab-cdef-0123-456789abcdef",
"typeURI": "data_plane"
},
"severity": "WARNING"
},
"attachments": [
"source": {
"namespace": "dpp-proxy",
"pod": "dynamic-proxy-b48ad3d7-7b7c-431f-ad5d-633e5641c0cd-59c677blq52m"
},
{
"userId": "proxy-user",
"request": {
"query": "select * from library.authors"
},
"correlationId": "93429272-b9e7-4452-b799-dfeddf5e3a47",
"responseData": null,
"requestData": {
"transformations": [
{
"field": [
{
"catalog": "acmedb",
"schema": "library",
"table": "authors",
"field": "lastname"
}
],
"singleFieldTransformation": {
"retain": {}
}
},
{
"field": [
{
"catalog": "acmedb",
"schema": "library",
"table": "authors",
"field": "firstname"
}
],
"singleFieldTransformation": {
"regex": {
"regex": "[A-Za-z]{1,15}"
}
}
},
{
"field": [
{
"catalog": "acmedb",
"schema": "library",
"table": "authors",
"field": "id"
}
],
"singleFieldTransformation": {
"retain": {}
}
}
],
"filters": [
{
"field": {
"catalog": "acmedb",
"schema": "library",
"table": "authors",
"field": "firstname"
},
"expression": "firstname <> 'AAAAA'"
}
]
}
}
]
}Example of the Verbose Query Audit Event for a Metadata Request
{
"header": {
"id": "00f5fc6b-a1d2-4db1-be44-9d7360a25176",
"event": {
"eventTime": "2022-11-04T12:36:49.439Z",
"eventType": "ACTIVITY",
"action": "policy.resolution"
},
"initiator": {
"id": "proxy-user",
"typeURI": "User",
"tenant": "fff7a844-f732-4a65-8280-912eae8eb8d0"
},
"observer": {
"id": "target"
},
"status": {
"outcome": "SUCCESS"
},
"target": {
"id": "01234567-89ab-cdef-0123-456789abcdef",
"typeURI": "data_plane"
},
"severity": "WARNING"
},
"attachments": [
{
"userId": "proxy-user",
"request": {
"metadata": "get tables"
},
"correlationId": "93429272-b9e7-4452-b799-dfeddf5e3a47",
"requestData": {
"transformations": [
{
"field": [
{
"catalog": "acmedb",
"schema": "library",
"table": "authors",
"field": "firstname"
}
],
"singleFieldTransformation": {
"regex": {
"regex": "[A-Za-z]{1,15}"
}
}
},
{
"field": [
{
"catalog": "acmedb",
"schema": "library",
"table": "authors",
"field": "id"
}
],
"singleFieldTransformation": {
"retain": {}
}
},
{
"field": [
{
"catalog": "acmedb",
"schema": "library",
"table": "authors",
"field": "lastname"
}
],
"singleFieldTransformation": {
"retain": {}
}
}
],
"filters": [
{
"field": {
"catalog": "acmedb",
"schema": "library",
"table": "authors",
"field": "firstname"
},
"expression": "firstname <> 'AAAAA'"
}
]
}
}
]
}Logs for Possible Failures
There are also audit logs for some possible failures, such as when user authentication has failed:
{
"header": {
"id": "00f5fc6b-a1d2-4db1-be44-9d7360a25176",
"event": {
"eventTime": "2022-11-04T12:36:49.439Z",
"eventType": "ACTIVITY",
"action": "user.authentication"
},
"initiator": {
"id": "user@gmail.com",
"typeURI": "User",
"tenant": "fff7a844-f732-4a65-8280-912eae8eb8d0",
"sourceIp": null
},
"observer": {
"id": "target"
},
"status": {
"outcome": "FAILURE",
"reason": {
"message": "Received error while trying to authenticate username user@gmail.com: Server Error"
}
},
"target": {
"id": "01234567-89ab-cdef-0123-456789abcdef",
"typeURI": "data_plane"
},
"severity": "WARNING"
},
"attachments": [
]
}This log shows a policy retrieval failure.
{
"header": {
"id": "00f5fc6b-a1d2-4db1-be44-9d7360a25176",
"event": {
"eventTime": "2022-11-04T12:36:49.439Z",
"eventType": "ACTIVITY",
"action": "policy.resolution"
},
"initiator": {
"id": "proxy-user",
"typeURI": "User",
"tenant": "fff7a844-f732-4a65-8280-912eae8eb8d0",
"sourceIp": null
},
"observer": {
"id": "target"
},
"status": {
"outcome": "FAILURE",
"reason": {
"message": "Failed to retrieve policies. Internal server error"
}
},
"target": {
"id": "01234567-89ab-cdef-0123-456789abcdef",
"typeURI": "data_plane"
},
"severity": "WARNING"
},
"attachments": []
}