Implementation Guidelines

Overview

The Production Stack test benchmarks a realistic multi-service CRUD API deployment as a unit. The load generator sends HTTPS/h2 requests to the edge. Request flow depends on the path:

Architecture

Four services, plus the shared Postgres sidecar (external, managed by the benchmark script):

                  h2 + TLS
  ┌──────────┐   ─────────>    ┌─────────┐     auth_request        ┌──────────┐
  │  h2load  │   port 8443     │  edge   │  ──────────────────►   │ authsvc  │
  │          │                 │ (nginx) │                         │  (Rust)  │
  │          │   Authorization │         │  ◄── 200 + X-User-Id   │ JWT+HMAC │
  │          │   Bearer <jwt>  │         │                         └──────────┘
  └──────────┘                 │         │
                               │         │   /api/*
                               │         │   ────────────────►    ┌──────────┐
                               │         │   X-User-Id header     │  server  │
                               │         │                        │(framework│
                               │         │   /public/*            │          │
                               │         │   ────────────────►    │ L1 cache │
                               └─────────┘                        │ L2 Redis │
                                                                  │ Postgres │
                                                                  └────┬─────┘
                                                                       ▼
                                                                  ┌──────────┐
                                                                  │  cache   │
                                                                  │ (Redis)  │
                                                                  └──────────┘

Authentication — JWT HMAC-SHA256

Every /api/* request carries an Authorization: Bearer <jwt> header. The edge proxy fires an auth_request subrequest to the shared authsvc sidecar on every single API call — no caching, no shortcuts. authsvc:

1. Reads the Authorization: Bearer <token> header
2. Splits the JWT into header.payload.signature
3. Computes HMAC-SHA256(secret, header.payload) and compares with the provided signature
4. Base64-decodes the payload and extracts the sub claim (user_id)
5. Returns 200 OK with X-User-Id: <user_id> or 401 Unauthorized

The JWT secret is shared between the token generator (data/jwt-token.txt) and authsvc via the JWT_SECRET environment variable. The pre-generated token embeds {"sub":"42","name":"Alice Chen","exp":<future>}.

Key rules:

• Entries must use the shared authsvc binary unmodified. The Rust sidecar at frameworks/_shared/authsvc/ is the same for every framework entry. Do not reimplement JWT verification in the framework’s language.
• No auth caching at the edge. No proxy_cache on the /_auth location. Every /api/* request must trigger a real HMAC-SHA256 verification. The CPU cost of this verification is part of what the test measures.
• No auth logic in the framework. The framework receives requests with a trusted X-User-Id header set by the edge after successful JWT verification. The framework never parses JWTs, never validates tokens, never touches the Authorization header.

Caching

The framework must implement cache-aside on /api/items reads: check cache first, fall through to Postgres on miss, populate cache with a short TTL (≤1 second). On writes, the framework must invalidate the cache entry so the next read sees fresh data.

How the caching is implemented is entirely the framework’s choice. Examples:

• Two-tier L1+L2: in-process memory cache + Redis (e.g. .NET HybridCache, Rails ActiveSupport::Cache with memory + Redis). Highest performance because hot keys never leave the process.
• Redis-only: every cache-aside read goes to Redis. Simpler but slower — Redis single-thread becomes the ceiling at high rps.
• In-process only: no Redis for caching, just a local hashmap with TTL. Works but loses the shared cache on multi-instance deployments.
• Framework-native cache: whatever the framework ships (Django cache, Spring Cache, etc.).

The .NET reference entry uses HybridCache (L1 in-process + L2 Redis) because that’s the idiomatic .NET 9+ pattern. Other frameworks should use whatever caching approach is natural and documented for their ecosystem. The test measures how well the framework + its cache choice perform together — the cache strategy is part of the competition.

Cache-aside pattern

GET /api/items/{id}:

1. Check L1 (in-process memory) by key item:{id}
2. L1 miss → check L2 (Redis via the distributed cache abstraction)
3. L2 miss → SELECT ... FROM items WHERE id = $1 in Postgres
4. Populate both L1 and L2 with the result, TTL ≤ 1 second
5. Return JSON with X-Cache: HIT or X-Cache: MISS header

POST /api/items/{id}:

1. Parse JSON body (name, price, quantity)
2. UPDATE items SET ... WHERE id = $1 in Postgres
3. Invalidate cache key item:{id} via the abstraction’s RemoveAsync / equivalent (must clear both L1 and L2)
4. Return 204 No Content

GET /api/me:

1. Read X-User-Id header (set by edge from authsvc JWT verification)
2. Cache-aside lookup on user:{id} with TTL ≤ 30 seconds
3. Miss → SELECT id, name, email, plan FROM users WHERE id = $1
4. Return user JSON

Cache TTLs

What the cache hierarchy looks like under load

With 10,000 unique item IDs at ~250K rps (50% items):

This is a real three-tier cache hierarchy under real load. Every layer does measurable work.

Endpoint responsibilities

Workload

Two h2load instances run in parallel against the same compose stack:

Reads (20,000 URIs in production-stack-reads.txt, round-robin):

Writes (20 URIs in production-stack-writes.txt, round-robin, with JSON body):

The reads instance uses -c $conns -m 16. The writes instance uses -c $(conns/8) -m 8. Both carry Authorization: Bearer <jwt> so all /api/* requests authenticate successfully.

IDs 1–20 are targeted by both reads and writes, creating realistic cache churn on those entries. IDs 21–10,000 are read-only with stable cache behavior.

Docker Compose

Every production-stack entry ships compose.production-stack.yml with exactly four services:

Required compose settings

Same as Gateway-64: network_mode: host, pinned cpuset, seccomp:unconfined, and memlock/nofile ulimits on all four services.

Environment variables

CPU allocation

64 logical CPUs (32 physical + 32 SMT siblings) split across four services. The split is the entry’s choice, but the reference entry’s empirically-tuned split is:

Important tuning finding: authsvc scales linearly with cores (HMAC-SHA256 is embarrassingly parallel) BUT stealing cores from server or edge to give to authsvc often makes overall throughput worse because the downstream services can’t keep up. The optimal split is where edge, authsvc, and server are all near their utilization ceiling simultaneously.

Database

Two tables, both seeded by data/pgdb-seed.sql:

items — 100,000 rows, used by /api/items/{id}:

CREATE TABLE items (
    id INTEGER PRIMARY KEY,
    name TEXT, category TEXT, price INTEGER, quantity INTEGER,
    active BOOLEAN, tags JSONB, rating_score INTEGER, rating_count INTEGER
);

users — 4 rows, used by /api/me:

CREATE TABLE users (
    id INTEGER PRIMARY KEY,
    name TEXT, email TEXT, plan TEXT
);

What this test measures

• Full production stack throughput with real JWT auth, real cache hierarchy, real DB writes
• Framework cache abstraction efficiency — HybridCache, Rails.cache, Django cache, etc. under L1/L2 pressure with 10K-item working set
• JWT verification cost — every /api/* request does real HMAC-SHA256 crypto at the edge
• Cache-aside correctness — reads populate cache on miss, writes invalidate, TTL cycles naturally
• CRUD write path — JSON body parsing, Postgres UPDATE, cache invalidation
• CPU allocation strategy — how entries balance edge/auth/server on a fixed 64-CPU budget

Parameters