AWS RDS Proxy: Connection Pooling for Lambda and Serverless Workloads

The problem RDS Proxy solves is simple to describe and expensive to ignore: Lambda functions don’t maintain persistent connections. Every cold start opens a new database connection. At moderate scale — say, 200 concurrent Lambda invocations hitting a PostgreSQL instance — you have 200 simultaneous connection attempts. PostgreSQL’s max_connections on a db.t3.medium is 510. Divide those between your application, admin connections, and headroom, and 200 concurrent Lambdas will saturate a medium-tier RDS instance. The database starts rejecting connections; your Lambdas fail with too many connections; users see errors.

RDS Proxy sits between your application and the database, maintaining a pool of persistent connections to RDS. Your Lambdas connect to the proxy (fast, lightweight), and the proxy multiplexes those onto a smaller set of real database connections. A proxy running 100 connections to PostgreSQL can serve thousands of simultaneous Lambda requests without the database seeing more than 100 connections.

What RDS Proxy Does (and Doesn’t Do)

RDS Proxy provides connection pooling and connection management. It maintains long-lived connections to the database, authenticates clients via IAM or Secrets Manager, and handles failover during RDS Multi-AZ switchovers by routing new connections to the new primary automatically.

It supports MySQL (RDS and Aurora), PostgreSQL (RDS and Aurora), and SQL Server (RDS only). MariaDB is supported through the MySQL-compatible endpoint.

What it doesn’t do: query caching, read/write splitting (it supports read-only replicas via a reader endpoint, but doesn’t automatically route SELECTs to replicas — your application code still has to target the reader endpoint explicitly), or query optimization.

The proxy is not necessary for every workload. EC2-based applications with persistent connection pools (HikariCP, pgBouncer, connection pool middleware) don’t benefit — they already manage connections efficiently. The proxy is most valuable for Lambda, ECS tasks with short lifespans, and any architecture where connection churn is inherent.

Creating an RDS Proxy

RDS Proxy authenticates to the database using credentials stored in Secrets Manager. No passwords in environment variables or parameter store — the proxy retrieves credentials from Secrets Manager automatically.

# First, create the secret for RDS credentials
SECRET_ARN=$(aws secretsmanager create-secret \
  --name rds/my-postgres-credentials \
  --secret-string '{
    "username": "myapp",
    "password": "your-secure-password",
    "engine": "postgres",
    "host": "my-postgres.abc123.us-east-1.rds.amazonaws.com",
    "port": 5432,
    "dbname": "myappdb"
  }' \
  --query 'ARN' \
  --output text)

# Create IAM role for the proxy to access Secrets Manager
cat > /tmp/proxy-trust-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"Service": "rds.amazonaws.com"},
    "Action": "sts:AssumeRole"
  }]
}
EOF

PROXY_ROLE_ARN=$(aws iam create-role \
  --role-name RDSProxyRole \
  --assume-role-policy-document file:///tmp/proxy-trust-policy.json \
  --query 'Role.Arn' \
  --output text)

# Grant proxy role access to the secret
aws iam put-role-policy \
  --role-name RDSProxyRole \
  --policy-name SecretsManagerAccess \
  --policy-document "{
    \"Version\": \"2012-10-17\",
    \"Statement\": [{
      \"Effect\": \"Allow\",
      \"Action\": [\"secretsmanager:GetSecretValue\", \"kms:Decrypt\"],
      \"Resource\": [\"$SECRET_ARN\", \"arn:aws:kms:us-east-1:123456789012:key/your-kms-key\"]
    }]
  }"

# Create the proxy
PROXY_ARN=$(aws rds create-db-proxy \
  --db-proxy-name my-postgres-proxy \
  --engine-family POSTGRESQL \
  --auth '[{
    "AuthScheme": "SECRETS",
    "SecretArn": "'"$SECRET_ARN"'",
    "IAMAuth": "DISABLED"
  }]' \
  --role-arn $PROXY_ROLE_ARN \
  --vpc-subnet-ids subnet-private-1a subnet-private-1b \
  --vpc-security-group-ids sg-rds-proxy \
  --require-tls \
  --idle-client-timeout 1800 \
  --debug-logging false \
  --query 'DBProxy.DBProxyArn' \
  --output text)

echo "Proxy ARN: $PROXY_ARN"

# Register target (the RDS instance or Aurora cluster)
aws rds register-db-proxy-targets \
  --db-proxy-name my-postgres-proxy \
  --db-instance-identifiers my-postgres-instance

RequireTLS: true enforces encrypted connections between your application and the proxy. Always enable this — the proxy-to-database connection is already encrypted (RDS enforces TLS), and you want end-to-end encryption including the application-to-proxy leg.

IdleClientTimeout: 1800 closes client connections that have been idle for 30 minutes. Lambda functions that error out without cleanly closing their connections would otherwise hold proxy connections open indefinitely.

# Get the proxy endpoint (use this instead of the RDS endpoint in your app)
aws rds describe-db-proxies \
  --db-proxy-name my-postgres-proxy \
  --query 'DBProxies[0].Endpoint'

Your application connects to the proxy endpoint exactly as it would the RDS endpoint — same port, same PostgreSQL wire protocol, same credentials. The connection string changes; nothing else does.

IAM Authentication

For tighter security, enable IAM authentication on the proxy. Applications authenticate using AWS SigV4 tokens instead of database passwords — no password stored anywhere in the application environment.

# Create proxy with IAM auth required
aws rds create-db-proxy \
  --db-proxy-name my-secure-proxy \
  --engine-family POSTGRESQL \
  --auth '[{
    "AuthScheme": "SECRETS",
    "SecretArn": "'"$SECRET_ARN"'",
    "IAMAuth": "REQUIRED"
  }]' \
  --role-arn $PROXY_ROLE_ARN \
  --vpc-subnet-ids subnet-private-1a subnet-private-1b \
  --vpc-security-group-ids sg-rds-proxy \
  --require-tls

# IAM policy for Lambda to authenticate to the proxy
aws iam put-role-policy \
  --role-name MyLambdaRole \
  --policy-name RDSProxyConnect \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": "rds-db:connect",
      "Resource": "arn:aws:rds-db:us-east-1:123456789012:dbuser:prx-abc123/myapp"
    }]
  }'

In your Lambda function, generate an authentication token and use it as the password:

import boto3
import psycopg2

def get_db_connection():
    client = boto3.client('rds')
    
    # Generate auth token (valid for 15 minutes)
    token = client.generate_db_auth_token(
        DBHostname='my-secure-proxy.proxy-abc123.us-east-1.rds.amazonaws.com',
        Port=5432,
        DBUsername='myapp',
        Region='us-east-1'
    )
    
    conn = psycopg2.connect(
        host='my-secure-proxy.proxy-abc123.us-east-1.rds.amazonaws.com',
        port=5432,
        database='myappdb',
        user='myapp',
        password=token,
        sslmode='require'
    )
    return conn

# Cache connection at module level for warm Lambda reuse
# Connection is reused across warm invocations
_connection = None

def get_cached_connection():
    global _connection
    if _connection is None or _connection.closed:
        _connection = get_db_connection()
    return _connection

def handler(event, context):
    conn = get_cached_connection()
    with conn.cursor() as cur:
        cur.execute("SELECT count(*) FROM orders WHERE status = 'pending'")
        count = cur.fetchone()[0]
    return {'pending_orders': count}

The module-level _connection is the key pattern. On a cold start, get_cached_connection() creates a new connection to the proxy. On warm invocations, it reuses the existing connection. The proxy handles the actual database connection — your Lambda’s “connection” to the proxy is lightweight, and the proxy’s connection to PostgreSQL is long-lived and pooled.

Connection Pooling Behavior

Understanding how the proxy pools connections helps you tune MaxConnectionsPercent correctly. The proxy maintains a pool of database connections, up to a percentage of the database’s max_connections.

# Update proxy target group settings
aws rds modify-db-proxy-target-group \
  --db-proxy-name my-postgres-proxy \
  --target-group-name default \
  --connection-pool-config '{
    "MaxConnectionsPercent": 50,
    "MaxIdleConnectionsPercent": 25,
    "ConnectionBorrowTimeout": 120,
    "SessionPinningFilters": ["EXCLUDE_VARIABLE_SETS"]
  }'

MaxConnectionsPercent: 50 means the proxy uses at most 50% of the database’s max_connections. On a db.t3.medium PostgreSQL (max_connections ≈ 510), that’s 255 proxy-to-database connections. Your applications can have thousands of simultaneous connections to the proxy; the proxy maps those onto at most 255 real database connections.

ConnectionBorrowTimeout: 120 — if no database connection is available in the pool for 120 seconds, the proxy returns an error. This prevents indefinite queuing. Tune this based on your application’s tolerance for database wait time.

SessionPinningFilters: ["EXCLUDE_VARIABLE_SETS"] — connection pinning is the gotcha most people hit. When a client issues certain session-level commands (SET LOCAL, BEGIN for explicit transactions, temporary tables, prepared statements), the proxy pins that client to a specific database connection for the duration of the session. Pinning reduces multiplexing efficiency. EXCLUDE_VARIABLE_SETS tells the proxy to ignore SET commands for non-critical variables when deciding whether to pin — reduces unnecessary pinning from ORMs that send SET commands on connection initialization.

Failover Behavior

One of the most practical benefits of RDS Proxy: during a Multi-AZ failover, the proxy reconnects to the new primary automatically. Without a proxy, a Multi-AZ failover causes 20-30 seconds of connection errors as the DNS record updates and application connection pools drain and reconnect. With the proxy, failover is typically 20-30 seconds total — but during that window, the proxy queues incoming requests rather than rejecting them, then drains the queue once the new primary is available.

# Monitor proxy state during failover testing
aws rds describe-db-proxy-targets \
  --db-proxy-name my-postgres-proxy \
  --query 'Targets[].{State:TargetHealth.State,Reason:TargetHealth.Reason,Endpoint:Endpoint}'

Target states: AVAILABLE (healthy, accepting connections), UNAVAILABLE (unreachable), REGISTERING (newly added), DEREGISTERING (being removed). During failover, the primary target transitions through UNAVAILABLE before the new primary becomes AVAILABLE.

Pricing

RDS Proxy costs $0.015 per vCPU per hour of the associated RDS instance or Aurora cluster. For a db.t3.medium (2 vCPU): 2 × $0.015 = $0.03/hour = $21.60/month. For a db.r6g.4xlarge (16 vCPU): 16 × $0.015 = $0.24/hour = $172.80/month.

The proxy runs in your VPC with multi-AZ redundancy built in. No additional charge for the Multi-AZ redundancy of the proxy itself.

For Lambda-heavy architectures, the proxy cost is usually justified by what you avoid: RDS scaling triggered by connection exhaustion, retries on connection errors, and the complexity of managing max_connections limits. For workloads where connection churn isn’t a problem (persistent EC2-based applications with their own connection pools), adding a proxy adds cost and a network hop without clear benefit.

Connection pooling at the application layer (PgBouncer, HikariCP, SQLAlchemy’s connection pool) remains the right choice for traditional server-based applications. RDS Proxy is specifically the right tool for ephemeral compute — Lambda, short-lived ECS tasks, and anything else that can’t maintain long-lived connections.

For the database tier this proxy fronts, the DynamoDB single-table design guide covers the alternative approach of using a connection-less managed database that scales without connection limits. For monitoring database performance through the proxy, AWS CloudWatch deep dive covers the RDS and proxy metrics worth alerting on.