Zero Trust Backend Architecture — Never Trust, Always Verify Between Services

Introduction

Zero trust means: never trust, always verify—even within your own infrastructure. The assumption that internal networks are secure is dead. Compromised containers, rogue services, lateral movement attacks, and insider threats mean every service-to-service call must be authenticated, authorized, and encrypted.

This post covers mTLS with SPIFFE certificates, service identity, network policies, JWT service tokens, HMAC request signing, and Vault for dynamic credentials.

mTLS Between Microservices with SPIFFE
Service Identity and SPIFFE IDs
Network Policy: Default-Deny + Service Allowlist
JWT Service Tokens
Request Signing with HMAC
Audit Logging for Cross-Service Calls
Vault for Dynamic Database Credentials
Checklist
Conclusion

mTLS Between Microservices with SPIFFE

SPIFFE (Secure Production Identity Framework for Everyone) provides automatic certificate management:

import * as tls from 'tls';
import * as fs from 'fs';
import express from 'express';
import https from 'https';

interface SPIFFEIdentity {
  spiffeID: string; // spiffe://example.com/service/payment-api
  certificate: string;
  key: string;
  trustBundle: string;
}

class SPIFFEClient {
  private identity: SPIFFEIdentity;

  constructor(identity: SPIFFEIdentity) {
    this.identity = identity;
  }

  createSecureAgent(): https.Agent {
    return new https.Agent({
      cert: this.identity.certificate,
      key: this.identity.key,
      ca: this.identity.trustBundle,
      rejectUnauthorized: true,
    });
  }

  async callService(
    targetSpiffeID: string,
    url: string
  ): Promise<any> {
    const agent = this.createSecureAgent();

    try {
      const response = await fetch(url, { agent });
      return response.json();
    } catch (error) {
      console.error(`Failed to call ${targetSpiffeID}:`, error);
      throw error;
    }
  }
}

// Initialize SPIFFE identity from workload API
const loadSPIFFEIdentity = async (): Promise<SPIFFEIdentity> => {
  // In production, use SPIRE agent to fetch identity
  // For demo, load from files
  return {
    spiffeID: 'spiffe://example.com/service/api-gateway',
    certificate: fs.readFileSync('/var/run/secrets/workload.crt', 'utf-8'),
    key: fs.readFileSync('/var/run/secrets/workload.key', 'utf-8'),
    trustBundle: fs.readFileSync('/var/run/secrets/ca.crt', 'utf-8'),
  };
};

// Server setup with mTLS
const setupMTLSServer = async (
  port: number
): Promise<express.Express> => {
  const identity = await loadSPIFFEIdentity();

  const options: tls.SecureContextOptions = {
    cert: identity.certificate,
    key: identity.key,
    ca: identity.trustBundle,
    requestCert: true,
    rejectUnauthorized: true,
  };

  const app = express();

  // Verify client certificate
  app.use((req: any, res, next) => {
    if (!req.client.authorizationError) {
      const cert = req.client.getPeerCertificate();
      if (cert && cert.subject) {
        const clientSpiffeID = cert.subjectAltName
          ?.split(',')
          .find((san) => san.includes('spiffe://'))
          ?.split('URI:')[1];

        if (!clientSpiffeID) {
          return res.status(401).json({ error: 'Invalid SPIFFE ID' });
        }

        req.clientSpiffeID = clientSpiffeID;
      }
    }
    next();
  });

  app.post('/api/process', (req, res) => {
    console.log(`Call from: ${req.clientSpiffeID}`);
    res.json({ success: true });
  });

  https.createServer(options, app).listen(port);
  console.log(`mTLS server listening on ${port}`);

  return app;
};

// Client usage
const callPaymentService = async () => {
  const identity = await loadSPIFFEIdentity();
  const client = new SPIFFEClient(identity);

  try {
    const result = await client.callService(
      'spiffe://example.com/service/payment-api',
      'https://payment-api:8443/api/charge'
    );
    console.log('Payment result:', result);
  } catch (error) {
    console.error('Payment call failed:', error);
  }
};

Service Identity and SPIFFE IDs

Structure identity hierarchies:

interface ServiceIdentity {
  spiffeID: string;
  namespace: string;
  service: string;
  cluster: string;
  environment: string;
}

class IdentityValidator {
  parseSpiffeID(spiffeID: string): ServiceIdentity | null {
    // spiffe://example.com/cluster/prod/namespace/payments/service/api
    const match = spiffeID.match(
      /^spiffe:\/\/([^/]+)\/cluster\/([^/]+)\/namespace\/([^/]+)\/service\/([^/]+)$/
    );

    if (!match) return null;

    return {
      spiffeID,
      cluster: match[2],
      namespace: match[3],
      service: match[4],
      environment: this.inferEnvironment(match[2]),
    };
  }

  private inferEnvironment(cluster: string): string {
    if (cluster.startsWith('prod')) return 'production';
    if (cluster.startsWith('stag')) return 'staging';
    return 'development';
  }

  isServiceAllowed(
    source: ServiceIdentity,
    target: ServiceIdentity,
    policy: Map<string, string[]>
  ): boolean {
    const allowedTargets = policy.get(source.service) || [];
    return allowedTargets.includes(target.service);
  }
}

// Service mesh policy example
const servicePolicy = new Map([
  ['api-gateway', ['payment-api', 'user-api', 'product-api']],
  ['payment-api', ['database', 'vault', 'payment-processor']],
  ['user-api', ['database', 'vault']],
  ['product-api', ['database', 'cache']],
]);

const validator = new IdentityValidator();

// Verify request
const verifyServiceCall = (req: any): boolean => {
  const clientSpiffeID = req.clientSpiffeID;
  const source = validator.parseSpiffeID(clientSpiffeID);

  if (!source) {
    console.error(`Invalid SPIFFE ID: ${clientSpiffeID}`);
    return false;
  }

  // Would check against actual policy here
  return true;
};

Network Policy: Default-Deny + Service Allowlist

Kubernetes network policies enforce zero trust:

# NetworkPolicy: Default deny all ingress
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
spec:
  podSelector: {}
  policyTypes:
  - Ingress

---
# Explicit allow: api-gateway can call payment-api
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-api-to-payment
spec:
  podSelector:
    matchLabels:
      app: payment-api
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: api-gateway
    ports:
    - protocol: TCP
      port: 8443

---
# Egress policy: payment-api can only call database and vault
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: payment-api-egress
spec:
  podSelector:
    matchLabels:
      app: payment-api
  policyTypes:
  - Egress
  egress:
  - to:
    - podSelector:
        matchLabels:
          app: postgres
    ports:
    - protocol: TCP
      port: 5432
  - to:
    - podSelector:
        matchLabels:
          app: vault
    ports:
    - protocol: TCP
      port: 8200
  - to:
    - namespaceSelector: {}
    ports:
    - protocol: UDP
      port: 53  # DNS

JWT Service Tokens

Short-lived, service-to-service authentication:

import jwt from 'jsonwebtoken';

interface ServiceToken {
  iss: string; // Issuer (service identity)
  sub: string; // Subject (target service)
  aud: string; // Audience (target service)
  iat: number; // Issued at
  exp: number; // Expiration (short, e.g., 5 minutes)
  jti: string; // JWT ID (unique, prevent replay)
  scopes: string[]; // Requested permissions
}

class ServiceTokenIssuer {
  private issuerKey: string;
  private issuer: string;

  constructor(issuerKey: string, issuer: string) {
    this.issuerKey = issuerKey;
    this.issuer = issuer;
  }

  issueToken(
    targetService: string,
    scopes: string[] = []
  ): string {
    const now = Math.floor(Date.now() / 1000);

    const payload: ServiceToken = {
      iss: this.issuer,
      sub: targetService,
      aud: targetService,
      iat: now,
      exp: now + 300, // 5 minutes
      jti: `jti-${Math.random().toString(36).substr(2, 9)}`,
      scopes,
    };

    return jwt.sign(payload, this.issuerKey, {
      algorithm: 'RS256',
      issuer: this.issuer,
      audience: targetService,
    });
  }
}

// Middleware to verify service token
const verifyServiceToken = (
  publicKey: string
) => {
  return (req: any, res: express.Response, next: express.NextFunction) => {
    const authHeader = req.headers.authorization;

    if (!authHeader?.startsWith('Bearer ')) {
      return res.status(401).json({ error: 'Missing token' });
    }

    const token = authHeader.substring(7);

    try {
      const decoded = jwt.verify(token, publicKey, {
        algorithms: ['RS256'],
        issuer: 'spiffe://example.com',
      }) as ServiceToken;

      req.service = {
        issuer: decoded.iss,
        scopes: decoded.scopes,
      };

      next();
    } catch (error) {
      res.status(403).json({ error: 'Invalid token' });
    }
  };
};

// Usage
const issuer = new ServiceTokenIssuer(
  fs.readFileSync('/secrets/service.key', 'utf-8'),
  'spiffe://example.com/service/api-gateway'
);

const token = issuer.issueToken(
  'spiffe://example.com/service/payment-api',
  ['charges:create', 'charges:read']
);

// When calling payment-api:
const response = await fetch('https://payment-api/api/charges', {
  headers: {
    'Authorization': `Bearer ${token}`,
  },
});

Request Signing with HMAC

Additional layer: sign requests with HMAC:

import crypto from 'crypto';

interface SignedRequest {
  timestamp: string;
  signature: string;
  nonce: string;
}

class RequestSigner {
  private sharedSecret: string;

  constructor(sharedSecret: string) {
    this.sharedSecret = sharedSecret;
  }

  signRequest(
    method: string,
    path: string,
    body: string = ''
  ): SignedRequest {
    const timestamp = new Date().toISOString();
    const nonce = crypto.randomBytes(16).toString('hex');

    const message = `${method}\n${path}\n${timestamp}\n${nonce}\n${body}`;

    const signature = crypto
      .createHmac('sha256', this.sharedSecret)
      .update(message)
      .digest('hex');

    return { timestamp, signature, nonce };
  }

  verifyRequest(
    method: string,
    path: string,
    body: string,
    signedRequest: SignedRequest
  ): boolean {
    const maxAge = 5 * 60 * 1000; // 5 minutes
    const requestTime = new Date(signedRequest.timestamp);

    if (Date.now() - requestTime.getTime() > maxAge) {
      return false; // Request too old
    }

    const message = `${method}\n${path}\n${signedRequest.timestamp}\n${signedRequest.nonce}\n${body}`;

    const expectedSignature = crypto
      .createHmac('sha256', this.sharedSecret)
      .update(message)
      .digest('hex');

    return crypto.timingSafeEqual(
      Buffer.from(signedRequest.signature),
      Buffer.from(expectedSignature)
    );
  }
}

// Add signature to request headers
const addRequestSignature = (
  method: string,
  path: string,
  body: string,
  signer: RequestSigner
) => {
  const signed = signer.signRequest(method, path, body);

  return {
    'X-Signature': signed.signature,
    'X-Timestamp': signed.timestamp,
    'X-Nonce': signed.nonce,
  };
};

// Middleware to verify signature
const verifyRequestSignature = (signer: RequestSigner) => {
  return (req: any, res: express.Response, next: express.NextFunction) => {
    const signature = req.headers['x-signature'];
    const timestamp = req.headers['x-timestamp'];
    const nonce = req.headers['x-nonce'];

    if (!signature || !timestamp || !nonce) {
      return res.status(401).json({ error: 'Missing signature' });
    }

    const body = req.body ? JSON.stringify(req.body) : '';
    const isValid = signer.verifyRequest(
      req.method,
      req.path,
      body,
      { signature, timestamp, nonce }
    );

    if (!isValid) {
      return res.status(403).json({ error: 'Invalid signature' });
    }

    next();
  };
};

Audit Logging for Cross-Service Calls

Track every service-to-service interaction:

interface AuditLog {
  timestamp: Date;
  source: string;
  target: string;
  method: string;
  path: string;
  status: number;
  duration: number;
  error?: string;
}

class AuditLogger {
  async logCall(log: AuditLog): Promise<void> {
    // Write to immutable audit log (e.g., append-only)
    await db.auditLogs.insert(log);

    // Alert on suspicious patterns
    if (log.status >= 400) {
      const recentErrors = await db.auditLogs.countErrors(
        log.source,
        log.target,
        5
      );

      if (recentErrors > 5) {
        await this.alert(
          `High error rate: ${log.source} -> ${log.target}`,
          'CRITICAL'
        );
      }
    }
  }

  private async alert(message: string, severity: string): Promise<void> {
    console.error(`[${severity}] ${message}`);
    // Send to monitoring system
  }
}

const auditLogger = new AuditLogger();

// Middleware to log all calls
const auditCallMiddleware = (req: any, res: express.Response, next: express.NextFunction) => {
  const startTime = Date.now();
  const source = req.clientSpiffeID || req.headers['x-forwarded-for'];

  res.on('finish', async () => {
    const duration = Date.now() - startTime;

    await auditLogger.logCall({
      timestamp: new Date(),
      source,
      target: req.hostname,
      method: req.method,
      path: req.path,
      status: res.statusCode,
      duration,
    });
  });

  next();
};

Vault for Dynamic Database Credentials

Database credentials that rotate automatically:

import { VaultClient } from '@vaultproject/client';

interface DatabaseCredentials {
  username: string;
  password: string;
  ttl: number;
  leaseId: string;
}

class DynamicCredentialManager {
  private vault: VaultClient;
  private role: string;

  constructor(vaultAddr: string, token: string, role: string) {
    this.vault = new VaultClient({ baseUrl: vaultAddr, token });
    this.role = role;
  }

  async getCredentials(): Promise<DatabaseCredentials> {
    const response = await this.vault.request({
      method: 'GET',
      path: `/v1/database/static-creds/${this.role}`,
    });

    return {
      username: response.data.data.username,
      password: response.data.data.password,
      ttl: response.data.lease_duration,
      leaseId: response.data.lease_id,
    };
  }

  async renewCredentials(leaseId: string): Promise<void> {
    await this.vault.request({
      method: 'PUT',
      path: `/v1/auth/token/renew/${leaseId}`,
    });
  }
}

// Connection pool with automatic credential refresh
class VaultAwarePool {
  private credManager: DynamicCredentialManager;
  private pool: any;
  private refreshInterval: NodeJS.Timer;

  constructor(
    vaultAddr: string,
    vaultToken: string,
    role: string,
    dbConfig: any
  ) {
    this.credManager = new DynamicCredentialManager(
      vaultAddr,
      vaultToken,
      role
    );
    this.setupPool(dbConfig);
    this.startRefresh();
  }

  private async setupPool(dbConfig: any): Promise<void> {
    const creds = await this.credManager.getCredentials();
    this.pool = new Pool({
      ...dbConfig,
      user: creds.username,
      password: creds.password,
    });
  }

  private startRefresh(): void {
    this.refreshInterval = setInterval(async () => {
      const creds = await this.credManager.getCredentials();
      // Refresh pool with new credentials (transparent to application)
      this.pool.options.password = creds.password;
    }, 3600000); // Every hour
  }

  query(sql: string, values: any[]): Promise<any> {
    return this.pool.query(sql, values);
  }
}

const credPool = new VaultAwarePool(
  process.env.VAULT_ADDR || 'http://vault:8200',
  process.env.VAULT_TOKEN || '',
  'payment-api',
  {
    host: 'postgres',
    port: 5432,
    database: 'payments',
  }
);

Checklist

Conclusion

Zero trust backend architecture requires mTLS for encryption, SPIFFE for identity, network policies for lateral movement prevention, JWT tokens for authentication, HMAC signatures for integrity, and Vault for dynamic secrets. Together, these layers ensure that even if one service is compromised, the attacker cannot easily move laterally or access other services' databases.