Deploying Polarity Server on AWS with RDS and Elasticache
Pre-requisites
AWS Security Groups
EC2 Instance (allows 80/tcp & 443/tcp inbound from appropriate networks)
Postgres Database (allows 5432/tcp inbound from EC2 instance security group)
Cache Security Group (allows 6379/tcp inbound from EC2 instance security group)
ElastiCache Subnet Group (as required based on the VPC selected for ElastiCache instances)
RDS DB Subnet Group (as required based on the VPC selected during RDS setup)
4x AWS ElastiCache Redis 6.x Parameter Groups
Session Cache
Integration Cache
Metrics Cache
PG Cache
4x AWS ElastiCache Redis 6.x clusters (configuration details below)
1x AWS RDS PostgreSQL v13.4-r1 database (configuration details below)
1x AWS EC2 instance (configuration details below)
Overview
While the entire Polarity platform is typically deployed onto a single EC2 instance you can also leverage Elasticache for Redis and RDS when deploying into an AWS environment. With this setup the EC2 instance runs the NGINX web proxy, REST API, and integration framework while caching and database storage is provided by Elasticache and RDS.

Security Groups
3 security groups are recommended:
Security Group Name
Protocol Allowed
Port Allowed
Source
ec2
TCP (HTTP)
80
Subnets for Authorized Polarity Clients
TCP (HTTPS)
443
Subnets for Authorized Polarity Clients
rds-polarity-database
TCP (Custom TCP)
5432
ec2 security group
ec-polarity-caches
TCP (Custom TCP)
6379
ec2 security group
ElastiCache
Parameter Groups
The 4 redis instances have specific configuration values that need to be applied. This requires creating four different ElastiCache Parameter Groups (session-cache, integration-cache, metrics-cache, and pg-cache) for redis6.x
.
The Postgres database also requires specific configuration values and 1 RDS Parameter Group (postgresql-rds) will need to be created and applied to the RDS instance.
Parameter Group
Parameter Name
Parameter Value
polarity-session-cache
maxmemory-policy
noeviction
polarity-integration-cache
maxmemory-policy
volatile-ttl
notify-keyspace-events
Ex
polarity-metrics-cache
maxmemory-policy
allkeys-lru
polarity-pg-cache
maxmemory-policy
allkeys-lru
polarity-rds
(these settings assume a database with 32GiB of memory)
work_mem (in kb)
32768
max_connections
200
Elasticache for Redis does not support RDB backups via the save
directive in the same way regular Redis does. In addition, the appendonly
parameter is not supported for modern versions of Redis. Instead, Elasticache for Redis supports daily backups via their backup and restore parameter. Further availability and reliability should be done via multi-AZ read replicas: Mitigating Failures - Amazon ElastiCache for Redis
Session Cache
Create a Redis 6 ElastiCache cluster for the session cache with the following settings:
Cluster Engine: Redis
Cluster Mode Enabled is unchecked
Engine Version: 6.x
Port: 6379
Parameter Group: Choose the polarity-session-cache parameter group you previously created.
Node Type: cache.t3.small
Replicas: 2
Multi-AZ: Enable if you have an appropriate subnet setup, otherwise do not check
Subnet Group: Select as appropriate
Security Group: Remove default and select the ec-polarity-caches security group you created earlier
Encryption in-transit: can optionally be enabled
Redis AUTH: must be checked if Encryption in-transit is enabled
Access Control Option: Redis AUTH Default user
Redis Auth Token: <Set a token of your choice>
POLARITY_SESSION_CACHE_PASSWORD
environment variable must be set on your polarity-server.env
file if Redis AUTH is enabled.
Integration Cache
Cluster Engine: Redis
Cluster Mode Enabled is unchecked
Engine Version: 6.x
Parameter Group: Choose the polarity-integration-cache parameter group you previously created.
Node Type: cache.m6g.large
Replicas: 2
Multi-AZ: Enable if you have an appropriate subnet setup, otherwise do not check
Subnet Group: Select as appropriate
Security Group: Remove default and select the ec-polarity-caches security group you created earlier
Encryption in-transit: can optionally be enabled
Redis AUTH: must be checked if Encryption in-transit is enabled
Access Control Option: Redis AUTH Default user
Redis Auth Token: <Set a token of your choice>
POLARITY_INTEGRATION_CACHE_PASSWORD
environment variable must be set on your polarity-server.env
file if Redis AUTH is enabled.
Metrics Cache
Cluster Engine: Redis
Cluster Mode Enabled is unchecked
Engine Version: 6.x
Parameter Group: Choose the polarity-metrics-cache parameter group you previously created.
Node Type: cache.m6g.large
Replicas: 2
Multi-AZ: Enable if you have an appropriate subnet setup, otherwise do not check
Subnet Group: Select as appropriate
Security Group: Remove default and select the ec-polarity-caches security group you created earlier
Encryption in-transit: can optionally be enabled
Redis AUTH: must be checked if Encryption in-transit is enabled
Access Control Option: Redis AUTH Default user
Redis Auth Token: <Set a token of your choice>
POLARITY_METRICS_CACHE_PASSWORD
environment variable must be set on your polarity-server.env
file if Redis AUTH is enabled.
PG Cache
Cluster Engine: Redis
Cluster Mode Enabled is unchecked
Engine Version: 6.x
Parameter Group: Choose the polarity-pg-cache parameter group you previously created.
Node Type: cache.t3.small
Replicas: 2
Multi-AZ: Enable if you have an appropriate subnet setup, otherwise do not check
Subnet Group: Select as appropriate
Security Group: Remove default and select the ec-polarity-caches security group you created earlier
Encryption in-transit: can optionally be enabled
Redis AUTH: must be checked if Encryption in-transit is enabled
Access Control Option: Redis AUTH Default user
Redis Auth Token: <Set a token of your choice>
POLARITY_PG_CACHE_PASSWORD
environment variable must be set on your polarity-server.env
file if Redis AUTH is enabled.
AWS RDS PostgreSQL
Create an RDS PostgreSQL database using the Standard Creation method with the following settings:
Engine Options
Engine Type: PostgreSQL
Version: 13.3-R1
Templates
Template: Production
Settings
DB instance identifier: customer specific
Master username: breach
Master password: specify or let RDS auto generate
DB Instance Size
DB instance class
Standard class: db.m5.2xlarge
Storage
Storage Type: Provisioned IOPS (SSD)
Allocated storage: 100GiB
Provisioned IOPS: 3000
Enable storage autoscaling: checked
Maximum storage threshold: 1000
Availability & durability
Multi-AZ deployment: create a standby instance
Connectivity
Virtual private cloud: customer specific
Subnet group: customer specific (must be created ahead of time)
Public Access: No
VPC Security Group: Choose existing
Remove default
Add rds-polarity-database
Database port: 5432
Database Authentication
Password authentication
Additional configuration
Database Options
Initial database name: breach
DB parameter group: polarity-rds
Backup/Encryption/Monitoring: Maintenance: customer specific
When “View Credential Details” opens, please ensure you save your database credentials in a safe location as you cannot retrieve them once you close the window.
Polarity Server
EC2 Instance
Create an EC2 instance for the Polarity Server with the following settings:
Instance AMI: “CentOS 7 (x86_64) - with Updates HVM”
Instance Type: m5.xlarge
EBS Storage: at least 40GB General Purpose SSD
Existing Security Group: ec2
Install Polarity Server
Pre-Installation Steps
Download Installation TGZ
Untar the installation file and change into the installation directory:
tar -xzvf polarity-server-4.1.0-web-4.1.0-el7.tgz
cd polarity-server-4.1.0-web-4.1.0
After untaring the upgrade package you will find the following files:
polarity-web-4.1.0-1.el7.noarch.rpm
polarity-server-4.1.0-1.el7.x86_64.rpm
polarity-metrics-cache-1.0.16+6.2.4-1.el7.x86_64.rpm
polarity-integration-cache-1.0.16+6.2.4-1.el7.x86_64.rpm
polarity-session-cache-1.0.16+6.2.4-1.el7.x86_64.rpm
polarity-pg-cache-1.0.16+6.2.4-1.el7.x86_64.rpm
polarity-nodejs-1.0.1+12.22.5-1.el7.x86_64.rpm
polarity-nodejs-devel-1.0.1+12.22.5-1.el7.x86_64.rpm
polarity-nodejs-npm-1.0.1+12.22.5-1.el7.x86_64.rpm
upgrade-to-polarity-server-4.1.0.sh
SHA256SUMS
We will not be using the upgrade-to-polarity-server-4.1.0.sh
script for this install
Installation Steps
Optional: Update System
yum clean all
yum update -y
Install Base Dependencies
yum install -y epel-release
yum install https://download.postgresql.org/pub/repos/yum/\
reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm -y
yum install wget ansible libselinux-python nginx gcc-c++ \
postgresql13-server postgresql13-contrib postgresql13-libs \
git ip4r13 python-psycopg2 policycoreutils-python \
libicu -y
Install Node RPMs
yum install polarity-nodejs-*.rpm -y
Install the Polarity Server. The Polarity Server install process can take up to 15 minutes. Please be patient while the RPM installs.
yum install polarity-server-4.1.0-1.el7.x86_64.rpm -y
After the RPM completes installation you’ll notice an error at the end indicating that the database migrations failed to run. This migration error is expected as we will not be using the local postgresql instance for our deployment.
/// EXPECTED ERROR
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] migrations:run: \
`./migration/scripts/init-db.sh && ./node_modules/.bin/ts-node \
--transpile-only ./node_modules/.bin/typeorm migration:run -t=false`
Stop the local copy of PostgreSQL as we will be using our RDS instance.
systemctl stop postgresql-13
systemctl disable postgresql-13
The local copy of PostgreSQL is required to as it is a requirement for the RPM. We will not be connecting to it however and can safely stop the service.
Install Polarity Web
yum install polarity-web-4.1.0-1.el7.noarch.rpm -y
Modify Cache and PostgreSQL Configuration
Open the environment file to edit connection settings for the database and caches.
vi /app/polarity-server/.env
Set the following environment variables:
POLARITY_INITIALIZE_REDIS_CONFIGS_ON_START=false
POLARITY_SESSION_CACHE_HOST=<your-amazon-polarity-session-cache-host>
POLARITY_SESSION_CACHE_PORT=6379
POLARITY_SESSION_CACHE_DB=0
POLARITY_SESSION_CACHE_TTL=2592000
POLARITY_INTEGRATION_CACHE_HOST=<your-amazon-polarity-integration-cache-host>
POLARITY_INTEGRATION_CACHE_PORT=6379
POLARITY_INTEGRATION_CACHE_DB=0
POLARITY_PG_CACHE_HOST=<your-amazon-polarity-pg-cache-host>
POLARITY_PG_CACHE_PORT=6379
POLARITY_PG_CACHE_DB=0
POLARITY_METRICS_REDIS_HOST=<your metrics cache host on aws>
POLARITY_METRICS_REDIS_PORT=6379
POLARITY_METRICS_REDIS_DB=0
POLARITY_DB_SUPERUSER_USER=breach
POLARITY_DB_SUPERUSER_PASSWORD=<RDS "breach" user password here>
POLARITY_DB_HOST=<RDS Endpoint Hostname Here>
POLARITY_DB_PORT=5432
POLARITY_DB_DATABASE=breach
POLARITY_DB_USER=polarity
POLARITY_DB_SCHEMA=polarity
POLARITY_DB_PASSWORD=<RDS "polarity" user password here>
# The following cache passwords are only required if you enabled
# Redis AUTH when standing up your Elasticache for Redis instances
POLARITY_INTEGRATION_CACHE_PASSWORD=<integration cache auth password>
POLARITY_METRICS_CACHE_PASSWORD=<metrics cache auth password>
POLARITY_PG_CACHE_PASSWORD=<pg cache auth password>
POLARITY_SESSION_CACHE_PASSWORD=<session cache password>
# Required only if the rds.force_ssl parameter was set in the polarity-rds
# parameter group
PGSSLMODE=require
Note that you must set POLARITY_DB_SUPERUSER_PASSWORD
to password set when standing up the RDS instance in AWS.
Note that you can set POLARITY_DB_PASSWORD
to any sufficiently random string
Next open up the Polarity Server config.js
file which will require modifications
vi /app/polarity-server/config/config.js
Set the following configuration values for the Caches
pufferfish.redisStateStore.host: "<your session cache host on aws>"
pufferfish.redisStateStore.port: 6379
pufferfish.redisStateStore.db: 2
postgres.database: "breach"
postgres.port: 5432
postgres.host: '<your RDS host>'
postgres.user: 'breach'
postgres.password: '<your RDS breach user password>'
postgres.schema: 'public'
Note that the we are using JSON dot notation to represent the nested JSON structure of the config file
Initialize RDS Database
Save the following script to a file called setup_rds_db.sh
#!/usr/bin/env bash
# setup_rds_db.sh
set -Eeo pipefail
source /app/polarity-server/.env
export PGPASSWORD="$POLARITY_DB_SUPERUSER_PASSWORD"
echo "start IS_INITIALIZED"
IS_INITIALIZED=$(psql -U "$POLARITY_DB_SUPERUSER_USER" \
-d "$POLARITY_DB_DATABASE" -h "$POLARITY_DB_HOST" -p \
"$POLARITY_DB_PORT" -t -c "SELECT to_regclass('polarity.users')")
if [[ "${IS_INITIALIZED}" =~ "users" ]]; then
echo "Polarity Server database has already been initialized"
exit 0
fi
echo "Initializing Polarity Server database"
echo "start create user"
psql -v ON_ERROR_STOP=1 -h "$POLARITY_DB_HOST" -p "$POLARITY_DB_PORT" \
--username "$POLARITY_DB_SUPERUSER_USER" --dbname postgres <<-EOSQL
CREATE ROLE "$POLARITY_DB_USER" WITH LOGIN PASSWORD '$POLARITY_DB_PASSWORD';
GRANT "$POLARITY_DB_USER" to "$POLARITY_DB_SUPERUSER_USER";
EOSQL
echo "start create schema"
psql -v ON_ERROR_STOP=1 -h "$POLARITY_DB_HOST" -p "$POLARITY_DB_PORT" \
--username "$POLARITY_DB_SUPERUSER_USER" --dbname "$POLARITY_DB_DATABASE" <<-EOSQL
CREATE SCHEMA polarity AUTHORIZATION "$POLARITY_DB_USER";
EOSQL
echo "start grant all"
psql -v ON_ERROR_STOP=1 -h "$POLARITY_DB_HOST" -p "$POLARITY_DB_PORT" \
--username "$POLARITY_DB_SUPERUSER_USER" --dbname "$POLARITY_DB_DATABASE" <<-EOSQL
GRANT ALL PRIVILEGES ON SCHEMA polarity to "$POLARITY_DB_USER";
EOSQL
psql -v -U "$POLARITY_DB_SUPERUSER_USER" -d "$POLARITY_DB_DATABASE" \
-h "$POLARITY_DB_HOST" -p "$POLARITY_DB_PORT" \
-f "/app/polarity-server/deploy/sql/install/setup-schema-v3.6.sql"
echo "Polarity Server database initialization complete."
Ensure the RDS setup script is executable:
chmod a+x setup_rds_db.sh
Execute the script:
./setup_rds_db.sh
Run the Polarity database migration script
cd /app/polarity-server
npm run migrations:run
License Installation
At this point you will need to intall your Polarity License provided to you by your Polarity support or customer success team. The license file will be named polarity.lic
.
Copy the polarity.lic
file to the license directory
cp polarity.lic /app/polarity-server/license/polarity.lic
Ensure the license file is owned by the polarityd
user
chown polarityd:polarityd /app/polarity-server/license/polarity.lic
Finally restart your server so the license is loaded
systemctl restart polarityd
You can now navigate to the polarity web interface by opening a browser (Chrome is recommended) and navigating to https://<your-polarity-server>
You can login with the default user admin
and the password PolarityServer2015!
Update FQDN in Polarity Config
By default the Polarity server will assume the FQDN for your server matches the server hostname. If this is not the case, you will need to modify the Polarity Server config file to set the appropriate config.
To modify the Polarity server config begin by opening the config file in an editor.
vi /app/polarity-server/config/config.js
Find the setting rest.fullyQualifiedDomainName
and set it to your FQDN (you can also set this to an IP address):
"rest": {
"fullyQualifiedDomainName": "your.fqdn.com"
}
Save the change and restart the server:
systemctl restart polarityd
Enabling SSL between Polarity Server and RDS
By default, the underlying postgresql connector will ignore errors related to untrusted certificates. To enforce Open the .env
file located at /app/polarity-server/.env
and add the environment variable:
PGSSLMODE=require
This will enable SSL connections. By default, node-pg
will be set to rejectUnauthorized: false
and thus will allow untrusted SSL connections to happen. Both polarity-server, typeorm, and Slonik use node-pg version 7 which has this behavior.
Starting in node-pg v8+ the default behavior is for rejectUnauthorized: true
which means the above env var along will not be enough to connect to most PostgreSQL servers. See release notes here https://github.com/brianc/node-postgres/blob/54b87523e29ea53379d7b9a26e45f83886f371af/CHANGELOG.md#pg800
In addition to turning SSL on via PGSSLMODE
we can provide a custom CA via the NODE_EXTRA_CA_CERTS
environment variable:
NODE_EXTRA_CA_CERTS=/etc/ssl/certs/rds-combined-ca-bundle.pem
This has been tested to work for node-pg
and typeorm
but has not been explicitly tested to work with slonik
. If you are running migrations via npm run migrations:run
then you need to pass this variable explicitly into the command as it is not automatically picked up by the dotenv
package the same way the Polarity server is.
NODE_EXTRA_CA_CERTS=rds-combined-ca-bundle.pem npm run migrations:run
Last updated