Enabling Source Analytics

Enabling Source Analytics Logging

To enable Polarity Source Analytics logging on the Polarity server you will need to edit the .env file, which is located at /app/polarity-server/.env .

sudo vi /app/polarity-server/.env

In the .env file you will need to edit and/or add the following line:

POLARITY_LOG_INTEGRATION_LOOKUPS=verbose

There are 4 different logging levels for lookups off, minimal, info, verbose. Each logging level is additive data wise. As an example info contains all of the data in minimal.

To leverage the Polarity Source Analytics integration you will want to set the level to verbose.

When enabled, a JSON log entry will be generated for each lookup performed by the server. There will be one log entry per integration per entity. The generated logs are newline delimited JSON.

After modifying the .env file you will need to restart the polarity server process for the change to take effect.

sudo systemctl restart polarityd

After the server restarts, you will find telemetry log entries appearing in the server's log file located at

/app/polarity-server/logs/polarity-server.log

By default, the polarity-server.log file is rotated daily as long as the file exceeds 10MB. The last 30 days of log files are kept.

Logging Levels

off

The default logging level is off which will not log any integration lookups

minimal

The minimal logging level will log the following fields:

timestamp

An ISO8601 compliant timestamp which represents the time at which the lookup occurred.

msgType

A static value of integration-lookup designed to make it easy to identify source analytic logging messages.

integrationId

The id of the integration that performed the lookup

hasResult

A boolean true or false indicating whether or not the lookup returned a result

isFromCache

A boolean true or false indicating whether or not the result was returned from the integration cache

isOnDemand

A boolean true or false indicating whether or not the lookup was an on-demand lookup

entityTypes

An array (list) of all entity types that the looked up entity matches

primary EntityType

The primary entity type (e.g., IPv4, md5, domain etc.)

info

Contains all the information logged at the minimal level but also includes the userId

userId

The immutable integer id of the user that performed the lookup

verbose

Contains all information logged at the minimala and info level but also includes the following entity information.

entityValue

The value of the entity that was looked up.

windowTitle

If the lookup was not an on-demand lookup, windowTitle will be the title of the window where the entity was extracted from via Polarity's OCR capabilities.

process

If the lookup was not an on-demand lookup (i.e., was a Stream of Highlight lookup), the process will be the name of the application process where the entity was extracted from via Polarity's OCR capabilities.

Note that both the windowTitle and process information are only collected if the user is running in Stream or Highlight mode.

Example Source Analytics Log Entry

The following is an example of a log entry generated when POLARITY_LOG_INTEGRATION_LOOKUPS is set to verbose

{
   "hasResult":true,
   "isFromCache":false,
   "integrationId":"cyberchef",
   "msgType":"integration-lookup",
   "isOnDemand":true,
   "primaryEntityType":"custom",
   "entityTypes":[
      "custom.encodedString",
      "IPv4",
      "IP"
   ],
   "entityValue":"8.8.8.8",
   "userId":1,
   "level":"info",
   "message":"Integration Lookup",
   "timestamp":"2021-02-09T21:51:25.358Z"
}