Sending Source Analytics to Splunk

The following guide walks through how to collect source analytic logs from your server using Splunk.

Enable Source Analytics

Prior to setting up collection of your source analytic (PSA) logs, please ensure that source analytics

Enabling Source Analytics

Configure Docker Logging for Splunk

Docker supports sending logs to Splunk via the Docker Splunk Logging driver. To configure driver you will need to make modifications to the docker-compose.yml located at /app/docker-compose.yml.

The following logger options should be provided under services -> app

    logging:
      driver: "splunk"
      options:
        splunk-url: "https://<SPLUNK_INSTANCE>:8088"
        splunk-insecureskipverify: "false"
        splunk-token: "<SPLUNK_HTTP_EVENT_COLLECTOR_TOKEN>"
        splunk-source: "polarity"
        splunk-index: "polarity"
        splunk-format: "raw"
        tag: ""

Logging Driver Options

Splunk URL

Path to your Splunk Enterprise, self-service Splunk Cloud instance, or Splunk Cloud managed cluster (including port and scheme used by HTTP Event Collector) in one of the following formats: https://your_splunk_instance:8088, https://input-prd-p-XXXXXXX.cloud.splunk.com:8088, or https://http-inputs-XXXXXXXX.splunkcloud.com.

Splunk Insecure Skip Verify

If you are running a self hosted Splunk instance with a self-signed certificate you may need to tell Docker to ignore self-signed certificate errors. This option should default to "false".

Splunk Token

Splunk HTTP Event Collector token used to authenticate to your Splunk instance.

Splunk Source

Optional Event Source. This can be set to polarity but other values can be used for your organization.

Splunk Index

The event index to send telemetry data to. This should be set to polarity or another index value specific to Polarity telemetry and logging data.

Splunk Format

The message format for logs sent to Splunk. This option must be set to raw. The raw splunk-format ensures that docker writes the logs exactly as they are produced by the server.

If you use the json format, the logs will be wrapped in JSON a second time and the actual telemetry data will become escaped JSON.

Splunk Tag

Set the tag to an empty string (""). Without this setting, Docker will prepend the default tag which is the container id. The prepended container id results in a log that is no longer valid JSON.

The above settings will ensure that the telemetry logs generated by the v5 server are in the exact same format as the v4 server. This means you can send the logs to the same index and in most cases use the same dashboards (see below for notes on dashboarding).

Restart Server

After making changes to the .env and docker-compose.yml you will need to restart the docker containers that run the server using the following command:

cd /app && ./down.sh && ./up.sh

Telemetry Log Format

The v5 server telemetry logs no longer include the property message="Integration Lookup" on telemetry logs. If you were using this to filter out telemetry logs from other logs we recommend switching to msgType="integration-lookup".

The msgType field is also set on v4 telemetry logs so using this property as a filter will identify all telemetry logs from both v4 and v5 servers.