New Audit Log

This feature is currently available in Beta version

📘

The new version of Audit log that is described in this article is active only on the private and exclusive instances. If you are using the shared instance, please refer to the old Audit log article.

As security is a crucial concern to Bloomreach, we consider it necessary to allow our clients to have an overview of all activities performed in the Bloomreach Engagement app. Especially when an activity is suspicious, we want you to be able to detect and investigate it. Audit logs, capturing and storing records of user activity within Bloomreach Engagement, enable you to do just that.

What is Audit log?

Audit logs are chronological records of user activity within the Bloomreach Engagement application. They capture audit trails that help in investigations of unauthorized access, data leak, or GDPR incidents, and they help with understanding alerts of repeated incidents and whether company-specific policies are met.

In simple terms, Audit Logs do that by answering the questions of Who? Did What? When? And Where?

  • Who? - The actor that performed an action. This can be public access, authenticated user, API key or Bloomreach Engagement's internal process.
  • Did What? - What action was performed can be identified by a unique combination of a request method and request path. Some records contain references to a specific resource and additional information within a service data.
  • When? - Each action has a precise timestamp. Time within the Bloomreach Engagement platform is synchronized using Network Time Protocol (NTP).
  • Where? - Actions are logged from different parts and scope of the application. The scope is one of: instance, account, or a project. Service name refers to one of many internal services. The screen where an action was performed by users from a browser can be accessed by the referrer URL.

Enabling and accessing the Audit log

To enable Audit log, you have to contact your CSM or Bloomreach Engagement support and, unless previously included within the original order form, a Letter of instruction must be signed. Enabling the Audit log usually takes up to 3 business days.

Access is managed by a CSM utilizing GSuite Google Groups (separate from the Access management within the Bloomreach Engagement application). Only the members of this group will be able to access the data. You will need Google Identities to use the Google Groups. These identities are either Google's user accounts or service accounts.

For security reasons (as a protection from any kind of manipulation or tampering with the audit log), there is no direct access from the Application, and users will be granted read-only access to Google Cloud Storage bucket containing exported Audit log data, which means the data itself cannot be directly edited.

How does Audit log work

Audit log tracks requests sent by application users. Results are stored in a separate GCS bucket and can be downloaded and processed by an automated system.

The Bloomreach Engagement app contains a component that verifies user access for all parts of the application. Every time a logged user performs an action, this component creates an event, which is then stored in specific storage. Some events are considered to be sensitive and are therefore supplemented with additional information (e.g. how many rows have been exported/imported etc.).

Buckets are named as [instance-auditlog-storage], where “instance” is the name of the instance in which the bucket is stored. The data in these buckets is organized in folders according to time in the following way: Year > Month > Day.

Files with audit log events are JSON one line. This means every event is noted on one line. Files are created every hour and each file contains new logs. Files are not updated after creation. Delayed records are written in a file with an incremental number suffix.

🚧

Files are available for download for at least 60 days. After this time period, they can be moved into our archive storage and are not accessible for download.

Understanding the Audit log

Correlating multiple records

Some actions may generate more records on different levels of the application. Different components add component-specific information to the logs as service data. The unique request ID is generated on the first touchpoint and then propagated to subsequent events.

Scope identification

All events contain a field that defines on which layer the event was created - connecting it with a specific project, account, or instance.

  • An action performed on a project level has the Scope type value “PROJECT” and Scope ID value is the project token
  • An action performed on an account level has the Scope type value “ACCOUNT” and Scope ID value is the account ID
  • An action performed on the instance level has the Scope type value “INSTANCE” and no Scope ID value

Correlating user activity

Use session ID to correlate and analyze the flow of user activity during a session. A new unique session ID is created for each login session and discarded after logout or session expiration. Methods that do not require authentication do not have a session ID.

Audit log record schema

Log records have a predefined schema and come in JSON format. Top-level schema is fixed which makes audit logs to be easily imported into all commonly used SIEMs (e.g. Splunk, Graylog).

Fixed schema fields are reliable, can be used as indexed columns, and shall be used for SIEM alerts. Drilling down, records may contain not fixed (untyped) fields that are meant to provide additional information, are useful during investigations, and may be used for less critical SIEM alerts.

Schema: Fixed fields

Fixed fields are always present. These include:

Field name

Description

timestamp

The timestamp of the operation, ISO 8601 datetime in UTC, e.g. 2020-07-09T11:30:57.239442245Z

request

method or path

status

HTTP status code, e.g. 403 for Forbidden actions

serviceName

The internal name of the service within Bloomreach Engagement architecture

scopeType

enum: INSTANCE, ACCOUNT, PROJECT

scopeID

A unique project or account identifier

requestID

A unique ID, useful for correlating individual records connected with single user action

📘

Scope-related fields (scopeType, scopeID) are present only for methods requiring authorization. Fields are not present for other methods which are public methods or require only authentication.

Schema: Optional fields

Optional fields are present only for some operations.

Field name

Description

authenticationInfo

identity: operations performed by an authenticated actor, e.g. read/update user profile, two-factor activation, logout

type: e.g. USER, SYSTEM_SERVICE_ACCOUNT, BASIC_AUTH

authorizationInfo

allowed: whether the action was allowed by an authorization check

permission: operations that require permission, e.g. customer data read

metadata

clientIP: a remote IP address from where request was called

host: hostname (domain) for external requests

referrer: browser reported document referrer, useful to find the origin screen

session: session ID for requests within authenticated user session

userAgent: rowser reported User-Agent

resource_id

Every operation on a specific resource with internal ID, e.g. customer, scenario, banner, experiment.

Useful for, e.g. filtering, alerts

Schema: Service Data

ServiceData types:

  • GenericServiceData
  • PolicyDelta
  • AnonymizationServiceData

GenericServiceData
Generic service data provide some additional information only for informational use cases. This information may be useful during investigations. Info field is not fixed and should not be used for critical use cases.

Field name

Description

@type

auditlog.GenericServiceData

info (untyped)

JSON encoded string, informational field, schema is not fixed and it's subject to change

version_id

Every operation on a specific resource with enabled versioning history, e.g. reports create/update, scenario create/update/start

{"@type":"auditlog.GenericServiceData","info":"{\"action_types\":[\"ab-split\",\"batch-webhook-action\",\"condition\",\"limit\",\"now-trigger\",\"on-event-trigger\",\"planned-trigger\",\"repeated-trigger\",\"send-email-action\",\"set-property-action\",\"wait-action\"],\"webhooks\":{\"endpoints\":[\"https://aud.app.exponea.dev/\"]},\"webhook_has_general_consent\":false,\"email_has_general_consent\":false}","versionID":"5f3b83344785e844fa76b0fd"}

PolicyChange

Field name

Description

@type

iam.PolicyUpdate

policyChanges

A list of policy changes.

member: user ID of affected user

roleId: role ID that was changed; e.g. "LegacyProjectAdmin"

type: change type, enum:

  • ADD: role has been granted
  • UPDATE: role has been updated
  • REMOVE: role has been revoked

oldExpireTIme: (optional) previously set role expiration timestamp

newExpireTime: (optional) newly set role expiration timestamp

updateTime

The timestamp of the operation

userID

User ID of the subject performing the operation

{"@type":"iam.PolicyUpdate","id":"5f3cdcc931b9028e985237c1","policyChanges":[{"member":"[email protected]","roleId":"PII","type":"ADD"}],"updateTime":"2020-08-19T08:03:21.529134238Z","userId":"[email protected]"}
{"@type":"iam.PolicyUpdate","id":"5f3633bc699b028f7121560b","policyChanges":[{"member":"[email protected]","roleId":"LegacyProjectAdmin","type":"REMOVE"},{"member":"[email protected]","roleId":"PII","type":"REMOVE"},{"member":"[email protected]","roleId":"LegacyAccountAdmin","type":"REMOVE"}],"updateTime":"2020-08-14T06:48:28.870266563Z","userId":"[email protected]"}
{"@type":"iam.PolicyUpdate","id":"5f3cf286a011be198cde4905","policyChanges":[{"member":"[email protected]","newExpireTime":"0001-01-01T00:00:00Z","oldExpireTime":"2020-08-22T22:00:00Z","roleId":"LegacyProjectAdmin","type":"UPDATE"}],"updateTime":"2020-08-19T09:36:06.174072194Z","userId":"[email protected]"}

AnonymizationServiceData

Field name

Description

@type

auditlog.AnonymizationServiceData

requests

A list of the requested IDs.

ids: A list of the external customer IDs.

*ID name = "_id" for Bloomreach Engagement's internal customer ID

{"authenticationInfo":{"identity":"44zrbjouu8jdio64zv6hli4ax0hklk0lkbn5gn4bd151epdffj2mhwj1lteta4tr","type":"BASIC_AUTH"},"authorizationInfo":{"allowed":true},"instanceName":"aud","logName":"audit.v1.Log","metadata":{"clientIP":"34.76.115.106","host":"aud.api.exponea.dev","userAgent":"PostmanRuntime/7.26.2"},"request":{"@type":"http","method":"POST","path":"/data/v2/projects/a25641b8-dd39-11ea-b199-ae02a0152881/customers/anonymize_bulk"},"serviceData":{"@type":"auditlog.AnonymizationServiceData","requests":[{"ids":[{"name":"registered","value":"1"}]},{"ids":[{"name":"_id","value":"5f3ea5a70d61525115e31e51"}]}]},"serviceName":"ext-api","status":200,"timestamp":"2020-08-20T16:50:41.801251Z"}