Chapter 10: Deployment Patterns and Operations

Overview

Production WSO2 deployments range from simple single-node setups to distributed, multi-region clusters. This chapter covers deployment topologies, Kubernetes deployment, CI/CD pipelines, and operational tasks.

Deployment Topologies

1. Single Node (All-in-One)

Everything runs on one server. Suitable for development, testing, or small workloads.

┌───────────────────────────────┐
│       WSO2 API Manager        │
│  ┌─────────┐  ┌────────────┐ │
│  │Publisher │  │Dev Portal  │ │
│  └─────────┘  └────────────┘ │
│  ┌─────────┐  ┌────────────┐ │
│  │Gateway  │  │Key Manager │ │
│  └─────────┘  └────────────┘ │
└──────────────┬────────────────┘
               │
         ┌─────┴─────┐
         │  Database  │
         └───────────┘

Pros: Simple, minimal infrastructure Cons: Single point of failure, limited scalability

2. Distributed Deployment

Separate control plane and data plane for scalability and isolation.

        Control Plane                    Data Plane
┌──────────────────────┐       ┌──────────────────────┐
│ Publisher Portal     │       │  Gateway Node 1      │
│ Developer Portal     │       │  Gateway Node 2      │
│ Key Manager          │       │  Gateway Node N      │
│ Traffic Manager      │       │                      │
└──────────┬───────────┘       └──────────┬───────────┘
           │                              │
           └──────────┬───────────────────┘
                      │
               ┌──────┴──────┐
               │  Databases   │
               │  ┌────────┐ │
               │  │apim_db │ │
               │  │shared_db│ │
               │  └────────┘ │
               └─────────────┘

deployment.toml (Gateway profile):

[server]
hostname = "gw.example.com"
server_role = "gateway-worker"

[apim.gateway.environment]
name = "Production"
type = "production"

# Point to control plane
[apim.key_manager]
service_url = "https://cp.example.com:9443/services/"

[apim.throttling]
service_url = "wss://cp.example.com:9443/"

3. Active-Active HA Deployment

Multiple instances of every component behind a load balancer.

              Load Balancer (L7)
                    │
        ┌───────────┼───────────┐
        │           │           │
   ┌────┴───┐  ┌───┴────┐  ┌──┴─────┐
   │ GW-1   │  │ GW-2   │  │ GW-3   │
   └────┬───┘  └───┬────┘  └──┬─────┘
        │          │           │
        └──────────┼───────────┘
                   │
           ┌───────┴───────┐
           │ DB Primary    │
           │ DB Replica    │
           └───────────────┘

Load Balancer Configuration (Nginx):

upstream apim_gateway {
    least_conn;
    server gw1.example.com:8243;
    server gw2.example.com:8243;
    server gw3.example.com:8243;
}

upstream apim_portal {
    ip_hash;
    server cp1.example.com:9443;
    server cp2.example.com:9443;
}

server {
    listen 443 ssl;
    server_name api.example.com;

    ssl_certificate /etc/ssl/api.example.com.crt;
    ssl_certificate_key /etc/ssl/api.example.com.key;

    location / {
        proxy_pass https://apim_gateway;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

server {
    listen 443 ssl;
    server_name portal.example.com;

    ssl_certificate /etc/ssl/portal.example.com.crt;
    ssl_certificate_key /etc/ssl/portal.example.com.key;

    location / {
        proxy_pass https://apim_portal;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

4. Multi-Region Deployment

Region A (Primary)              Region B (DR)
┌────────────────┐             ┌────────────────┐
│ CP + GW Cluster│             │ CP + GW Cluster│
└───────┬────────┘             └───────┬────────┘
        │                              │
   ┌────┴────┐                    ┌────┴────┐
   │ DB (RW) │ ──── Replication ──│ DB (RO) │
   └─────────┘                    └─────────┘

Kubernetes Deployment

Helm Charts

WSO2 provides official Helm charts for Kubernetes deployment.

# Add WSO2 Helm repo
helm repo add wso2 https://helm.wso2.com
helm repo update

# Install API Manager
helm install apim wso2/am-pattern-1 \
  --namespace wso2 \
  --create-namespace \
  --set wso2.subscription.username=<WSO2_USER> \
  --set wso2.subscription.password=<WSO2_PASS> \
  -f values.yaml

Custom values.yaml:

wso2:
  deployment:
    am:
      cp:
        replicas: 2
        resources:
          requests:
            memory: "2Gi"
            cpu: "1000m"
          limits:
            memory: "4Gi"
            cpu: "2000m"
      gateway:
        replicas: 3
        resources:
          requests:
            memory: "1Gi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "1000m"

  persistence:
    enabled: true
    storageClass: gp3

  ingress:
    gateway:
      hostname: api.example.com
    portal:
      hostname: portal.example.com

Micro Integrator on Kubernetes

Dockerfile:

FROM wso2/wso2mi:4.2.0

# Copy deployment artifacts
COPY carbonapps/*.car /home/wso2carbon/wso2mi-4.2.0/repository/deployment/server/carbonapps/

# Copy custom config
COPY deployment.toml /home/wso2carbon/wso2mi-4.2.0/conf/deployment.toml

Kubernetes Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mi-employee-service
  namespace: integration
spec:
  replicas: 3
  selector:
    matchLabels:
      app: mi-employee-service
  template:
    metadata:
      labels:
        app: mi-employee-service
    spec:
      containers:
        - name: mi
          image: registry.example.com/mi-employee-service:1.0.0
          ports:
            - containerPort: 8290
              name: http
            - containerPort: 9164
              name: management
          livenessProbe:
            httpGet:
              path: /liveness
              port: 9164
            initialDelaySeconds: 30
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /readiness
              port: 9164
            initialDelaySeconds: 20
            periodSeconds: 5
          resources:
            requests:
              memory: "512Mi"
              cpu: "250m"
            limits:
              memory: "1Gi"
              cpu: "500m"
          env:
            - name: JAVA_OPTS
              value: "-Xms512m -Xmx512m"
---
apiVersion: v1
kind: Service
metadata:
  name: mi-employee-service
  namespace: integration
spec:
  selector:
    app: mi-employee-service
  ports:
    - port: 8290
      targetPort: 8290
      name: http
    - port: 9164
      targetPort: 9164
      name: management

CI/CD Pipeline

Pipeline Stages

Code Commit → Build → Test → Package → Deploy (Dev) → Test → Deploy (Staging) → Deploy (Prod)

GitHub Actions Example

name: WSO2 MI Deployment

on:
  push:
    branches: [main]

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up JDK 17
        uses: actions/setup-java@v4
        with:
          java-version: '17'
          distribution: 'temurin'

      - name: Build Integration Project
        run: |
          cd my-integration
          mvn clean install -DskipTests

      - name: Run Integration Tests
        run: |
          cd my-integration
          mvn test

      - name: Build Docker Image
        run: |
          docker build -t registry.example.com/mi-service:${{ github.sha }} .

      - name: Push Docker Image
        run: |
          docker push registry.example.com/mi-service:${{ github.sha }}

      - name: Deploy to Staging
        run: |
          kubectl set image deployment/mi-service \
            mi=registry.example.com/mi-service:${{ github.sha }} \
            --namespace staging

      - name: Verify Deployment
        run: |
          kubectl rollout status deployment/mi-service \
            --namespace staging --timeout=120s

API Promotion with apictl

# Export from dev
apictl export api -n EmployeeAPI -v 1.0.0 -e dev

# Import to staging
apictl import api -f EmployeeAPI_1.0.0.zip -e staging --update

# Import to production
apictl import api -f EmployeeAPI_1.0.0.zip -e production --update

Environment-specific configs (api_params.yaml):

environments:
  - name: staging
    endpoints:
      production:
        url: https://staging-backend.example.com/api
      sandbox:
        url: https://staging-sandbox.example.com/api
    security:
      production:
        enabled: true
        type: basic
        username: staging_user
        password: staging_pass

  - name: production
    endpoints:
      production:
        url: https://prod-backend.example.com/api
    security:
      production:
        enabled: true
        type: basic
        username: prod_user
        password: prod_pass

JVM Tuning

Memory Configuration

# Set in <PRODUCT_HOME>/bin/api-manager.sh (or wso2server.sh)
# Or via JAVA_OPTS environment variable

# Development
export JAVA_OPTS="-Xms512m -Xmx1024m"

# Production Gateway
export JAVA_OPTS="-Xms2048m -Xmx2048m -XX:+UseG1GC -XX:MaxGCPauseMillis=200"

# Production Control Plane
export JAVA_OPTS="-Xms2048m -Xmx4096m -XX:+UseG1GC -XX:MaxGCPauseMillis=200"

Recommended Settings by Component

GC Tuning

# G1GC (recommended)
-XX:+UseG1GC
-XX:MaxGCPauseMillis=200
-XX:InitiatingHeapOccupancyPercent=45
-XX:G1HeapRegionSize=16m

# GC logging for analysis
-Xlog:gc*:file=/opt/wso2/logs/gc.log:time,uptime:filecount=5,filesize=20m

Database Optimization

Connection Pool Tuning

[database.apim_db.pool_options]
maxActive = 150
maxWait = 60000
minIdle = 10
testOnBorrow = true
validationQuery = "SELECT 1"
validationInterval = 30000
defaultAutoCommit = false

Database Sizing Guidelines

Database Maintenance

-- Purge expired tokens (run periodically)
DELETE FROM IDN_OAUTH2_ACCESS_TOKEN
WHERE TOKEN_STATE = 'EXPIRED'
  AND TIME_CREATED < DATE_SUB(NOW(), INTERVAL 30 DAY);

-- Purge old audit logs
DELETE FROM AM_API_AUDIT
WHERE CREATED_TIME < DATE_SUB(NOW(), INTERVAL 90 DAY);

-- Analyze tables for query optimization
ANALYZE TABLE IDN_OAUTH2_ACCESS_TOKEN;
ANALYZE TABLE AM_API;
ANALYZE TABLE AM_SUBSCRIPTION;

Backup and Recovery

What to Back Up

Backup Script

#!/bin/bash
BACKUP_DIR="/backup/wso2/$(date +%Y%m%d)"
mkdir -p "$BACKUP_DIR"

# Database backup
mysqldump -u wso2carbon -p apim_db > "$BACKUP_DIR/apim_db.sql"
mysqldump -u wso2carbon -p shared_db > "$BACKUP_DIR/shared_db.sql"

# Config backup
tar czf "$BACKUP_DIR/config.tar.gz" \
  /opt/wso2am/repository/conf/deployment.toml \
  /opt/wso2am/repository/resources/security/

# Artifact backup
tar czf "$BACKUP_DIR/artifacts.tar.gz" \
  /opt/wso2am/repository/deployment/server/

echo "Backup completed: $BACKUP_DIR"

Rolling Updates

Zero-Downtime Deployment

# Kubernetes rolling update
kubectl set image deployment/gateway \
  gateway=wso2/wso2am:4.2.1 \
  --namespace wso2

# Monitor rollout
kubectl rollout status deployment/gateway --namespace wso2

# Rollback if needed
kubectl rollout undo deployment/gateway --namespace wso2

Manual Rolling Update (Non-Kubernetes)

1. Remove GW-1 from load balancer
2. Stop GW-1
3. Update GW-1 binaries and configs
4. Start GW-1
5. Verify GW-1 health
6. Add GW-1 back to load balancer
7. Repeat for GW-2, GW-3, etc.

Health Checks

Gateway Health

# Liveness check
curl -k https://localhost:8243/services/Version

# Custom health endpoint (Micro Integrator)
curl -k https://localhost:9164/liveness
curl -k https://localhost:9164/readiness

Load Balancer Health Check

upstream apim_gateway {
    server gw1.example.com:8243;
    server gw2.example.com:8243;
    # Health check (Nginx Plus or use passive checks)
}

Key Takeaways

• Start with single-node for dev; use distributed deployment for production
• Separate control plane and data plane for independent scaling
• Kubernetes Helm charts simplify cloud-native deployment
• apictl enables CI/CD for API promotion across environments
• Tune JVM heap to match component role (gateway vs control plane)
• Database connection pools and periodic cleanup prevent performance degradation
• Automate backups for databases, configs, and keystores
• Rolling updates with proper health checks enable zero-downtime deployments

Next Steps

Continue to Chapter 11: Monitoring and Observability to learn about logging, analytics, and troubleshooting.