# resonate: full examples Complete code samples and walkthroughs from https://www.resonateperf.com/ ## Quick examples ### CLI ``` # POST with a templated body, gate on p95 in CI resonate hit https://api.example.com/orders \ -X POST \ -H 'Content-Type: application/json' \ -d '{"id": {{.Seq}}, "tok": "{{uuid}}"}' \ --rate 200 --workers 50 --duration 1m \ --expect-status 200,201 \ --assert 'success_rate>=0.99' \ --assert 'latency_p95<=500ms' ``` ### YAML ``` # scenario.yaml — run with: resonate run scenario.yaml # Illustrative: see `resonate schema` for the exact fields. name: checkout-flow duration: 2m rate: 100 steps: - name: login request: method: POST url: https://api.example.com/login body: '{"user": "demo"}' extract: token: json:$.token - name: cart request: url: https://api.example.com/cart headers: Authorization: Bearer {{.token}} ``` ### Go ``` import "github.com/jecklgamis/resonate" summary, err := resonate.NewScenario(). Get("http://localhost:8080/api/orders"). Rate(50).Workers(20). Duration(30 * time.Second). Run(ctx) if err != nil { log.Fatal(err) } summary.Print(os.Stdout) ``` ## Try it locally: scenarios ### Does it meet your SLO? You just wrote an API and have never load tested it. Is your /orders endpoint okay at 200 requests/sec? #### 1 · Write a local service with a hidden bottleneck (Python stdlib only) ```sh cat > svc.py << 'EOF' import time, threading from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler lock = threading.Lock() class H(BaseHTTPRequestHandler): def do_GET(self): with lock: # the bottleneck: ~50 req/s max time.sleep(0.02) self.send_response(200); self.send_header("Content-Length", "2") self.end_headers(); self.wfile.write(b"ok") def log_message(self, *a): pass ThreadingHTTPServer(("127.0.0.1", 18080), H).serve_forever() EOF ``` #### 2 · Terminal 1: start it ```sh python3 svc.py ``` #### 3 · Terminal 2: ask for far more than it can give, and gate on it ```sh resonate hit http://127.0.0.1:18080/orders \ --rate 200 --max-workers 200 --duration 10s \ --assert 'success_rate>=0.99' --assert 'latency_p95<=100ms' echo "exit code: $?" ``` Sample output: ``` Requests [total, rate] 638, 63.78/sec Duration [total] 10.003s Success [ratio] 66.46% Latencies [min, mean, p50, p75, p90, p95, p99, max] 28.63ms, 1.951831s, 1.926234s, 2.183135s, 2.204107s, 2.789212s, 8.002732s, 8.092297s Bytes [in, out] 848, 0 Status Codes [code:count] 200:424 Errors: [107] Get "http://127.0.0.1:18080/orders": dial tcp 127.0.0.1:18080: connect: operation timed out [107] Get "http://127.0.0.1:18080/orders": context deadline exceeded warning: target rate could not always be sustained — peak concurrency hit 200 in-flight iterations; raise --max-workers to open more headroom 2 of 2 assertion(s) failed: - success_rate: got 0.66, want >= 0.99 - latency_p95: got 2.789212159s, want <= 100ms error: 2 assertion(s) failed exit code: 3 ``` The run fails with exit code 3. Open report.html: p95 climbs into seconds while throughput stays flat near 60/s. Because --max-workers makes this an open-model test, the queueing shows up instead of being hidden. Remove the lock, rerun, and the assertions pass. ### Do your MCP sessions scale? You built an MCP server and agents are about to call it concurrently. Does your MCP server hold up under concurrent sessions, and which tool is the slow one? #### 1 · Terminal 1: run the example MCP server that ships with resonate (Go only, no other dependencies) ```sh git clone https://github.com/jecklgamis/resonate.git cd resonate make example-mcp-server # streamable HTTP on :8766 ``` #### 2 · Terminal 2: describe the session and the calls an agent makes (protocol: mcp) ```sh cat > mcp.yaml << 'EOF' protocol: mcp load: duration: 10s rate: 20 workers: 5 mcp: url: http://localhost:8766/mcp timeout: 5s calls: - tool: add arguments: {a: 2, b: 3} expect_result: json:structuredContent.result: "5" - tool: echo arguments: {message: "hello from vu {{.VU}}"} assertions: - metric: success_rate min: "0.999" EOF ``` #### 3 · Check one iteration, then load it ```sh resonate run mcp.yaml --dry-run resonate run mcp.yaml ``` Sample output: ``` Requests [total, rate] 448, 45.02/sec Duration [total] 9.952s Success [ratio] 100.00% Latencies [min, mean, p50, p75, p90, p95, p99, max] 72µs, 316µs, 290µs, 359µs, 474µs, 504µs, 2.689ms, 2.791ms Bytes [in, out] 51469, 47214 Steps: Step Requests Success p50 p95 p99 initialize 5 100.00% 2.744ms 2.791ms 2.791ms initialized 5 100.00% 157µs 176µs 176µs add 219 100.00% 339µs 508µs 617µs echo 219 100.00% 199µs 358µs 547µs Status Codes [code:count] 200:443, 202:5 ``` #### 4 · Model a realistic mix: agents that browse tools and agents that call them, at the same time ```sh cat > mcp-mix.yaml << 'EOF' protocol: mcp mcp: # shared by every scenario url: http://localhost:8766/mcp timeout: 5s scenarios: explorers: # agents browsing tools at a steady rate load: { duration: 10s, rate: 30, workers: 10 } mcp: calls: - list: tools - tool: echo arguments: {message: "hello from vu {{.VU}}"} calculators: # agents calling tools: 40 iterations a second, joining 2s late start_after: 2s load: { duration: 8s, rate: 40, workers: 20, max_workers: 100 } mcp: calls: - tool: add arguments: {a: 2, b: 3} expect_result: json:structuredContent.result: "5" - tool: solve_quadratic arguments: {a: 1, b: -3, c: 2} expect_result: json:structuredContent.real_roots: "2" assertions: - { scenario: calculators, step: add, metric: latency_p95, max: 50ms } - { metric: success_rate, min: "0.999" } EOF resonate run mcp-mix.yaml ``` Sample output: ``` Requests [total, rate] 1596, 159.94/sec Duration [total] 9.979s Success [ratio] 100.00% Latencies [min, mean, p50, p75, p90, p95, p99, max] 91µs, 361µs, 285µs, 394µs, 614µs, 1.04ms, 1.618ms, 3.523ms Steps: Step Requests Success p50 p95 p99 initialize 110 100.00% 491µs 2.008ms 2.331ms initialized 110 100.00% 268µs 1.477ms 1.69ms tools/list 329 100.00% 361µs 677µs 908µs echo 329 100.00% 189µs 362µs 403µs add 359 100.00% 320µs 857µs 1.609ms solve_quadratic 359 100.00% 187µs 560µs 1.381ms Scenario: explorers Requests [total, rate] 678, 68.02/sec Success [ratio] 100.00% Steps: Step Requests Success p50 p95 p99 initialize 10 100.00% 965µs 1.228ms 1.228ms initialized 10 100.00% 268µs 527µs 527µs tools/list 329 100.00% 361µs 677µs 908µs echo 329 100.00% 189µs 362µs 403µs Scenario: calculators (started after 2s) Requests [total, rate] 918, 115.09/sec Success [ratio] 100.00% Steps: Step Requests Success p50 p95 p99 initialize 100 100.00% 413µs 2.008ms 2.331ms initialized 100 100.00% 262µs 1.477ms 1.69ms add 359 100.00% 320µs 857µs 1.609ms solve_quadratic 359 100.00% 187µs 560µs 1.381ms ``` #### 5 · Optional: a tool that is meant to fail, and a new session every five iterations ```sh # add to mcp.calls: a tool error is a failure unless you expect it - tool: fail name: failing-tool expect_tool_error: true # add under load: retire each virtual user after 5 iterations, so every # retirement opens a new session and the handshake itself gets loaded iterations: 5 ``` resonate opens one MCP session per virtual user (initialize, then notifications/initialized) and makes the calls in order over it, sending the Mcp-Session-Id and protocol-version headers for you. Setup and every tool are separate rows in the Steps table, so you can see the handshake cost next to each tool and put a limit on a single tool (step: solve_quadratic). Replies as JSON or event streams both work. A tool that returns isError: true counts as a failure, so a server that answers but is broken cannot pass. Servers that only speak stdio are not supported; put an HTTP transport in front of them. ### What does a proxy cost? You are putting nginx in front of two backends. What does the proxy cost you, and what happens when a backend dies? #### 1 · Write two backends and an nginx config (brew install nginx) ```sh cat > backend.py << 'EOF' import sys from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler port, name = int(sys.argv[1]), sys.argv[2].encode() class Server(ThreadingHTTPServer): request_queue_size = 128 # the default of 5 drops connections under load class H(BaseHTTPRequestHandler): protocol_version = "HTTP/1.1" def do_GET(self): self.send_response(200); self.send_header("Content-Length", str(len(name))) self.end_headers(); self.wfile.write(name) def log_message(self, *a): pass Server(("127.0.0.1", port), H).serve_forever() EOF cat > nginx.conf << 'EOF' events {} http { upstream app { server 127.0.0.1:9001; server 127.0.0.1:9002; keepalive 32; } server { listen 127.0.0.1:8888; location / { proxy_pass http://app; proxy_http_version 1.1; proxy_set_header Connection ""; } } } EOF ``` #### 2 · Terminal 1: backend A ```sh python3 backend.py 9001 A ``` #### 3 · Terminal 2: backend B ```sh python3 backend.py 9002 B ``` #### 4 · Terminal 3: nginx, in the foreground ```sh nginx -c "$PWD/nginx.conf" -e /dev/stderr -g 'daemon off;' ``` #### 5 · Terminal 4: same load, direct vs through the proxy, then diff the reports ```sh mkdir direct proxied (cd direct && resonate hit http://127.0.0.1:9001/ --rate 500 --duration 10s) (cd proxied && resonate hit http://127.0.0.1:8888/ --rate 500 --duration 10s) resonate compare direct/report.json proxied/report.json ``` Sample output: ``` Metric Baseline Current Change requests 5500 5499 -0.0% rate 550.02 549.95 -0.0% success_rate 100.00% 100.00% +0.0% error_rate 0.00% 0.00% +0.0% latency_mean 1.351ms 2.35ms +73.9% latency_p50 232µs 373µs +60.7% latency_p75 259µs 414µs +59.6% latency_p90 369µs 623µs +68.8% latency_p95 9.732ms 20.906ms +114.8% latency_p99 21.823ms 28.508ms +30.6% ``` #### 6 · Terminal 4: run again, and Ctrl-C backend B (terminal 2) after a few seconds ```sh resonate hit http://127.0.0.1:8888/ --rate 200 --duration 10s ``` Sample output: ``` Requests [total, rate] 2199, 219.89/sec Duration [total] 10s Success [ratio] 100.00% Latencies [min, mean, p50, p75, p90, p95, p99, max] 151µs, 1.748ms, 413µs, 466µs, 687µs, 13.804ms, 18.498ms, 21.561ms Bytes [in, out] 2199, 0 Status Codes [code:count] 200:2199 ``` compare prints every metric side by side, so the cost of the extra hop is a number, not a feeling (add --fail-on to gate it in CI). Expect the median to move by a fraction of a millisecond and the tail to move more. In step 6, nginx notices the dead backend, retries the request on the other one and marks B as down, so success stays at 100% and any cost shows up only in the tail latency (compare p99 and max with step 5). The keepalive lines matter: without them nginx opens a new connection per request, and this toy Python backend drops some under load. ### Is your config live? Your API returns big JSON and Envoy passes it through untouched. After you change your Envoy config, how do you prove it took effect, and what does it cost? #### 1 · Write one backend that returns about 250 KB of JSON ```sh cat > backend.py << 'EOF' import json from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler # ~250 KB of repetitive JSON: the kind of API response that compresses well BODY = json.dumps([{"id": i, "name": f"item-{i}", "tags": ["alpha", "beta", "gamma"], "active": True} for i in range(3000)]).encode() class Server(ThreadingHTTPServer): request_queue_size = 128 # the default of 5 drops connections under load class H(BaseHTTPRequestHandler): protocol_version = "HTTP/1.1" def do_GET(self): self.send_response(200); self.send_header("Content-Type", "application/json") self.send_header("Content-Length", str(len(BODY))); self.end_headers(); self.wfile.write(BODY) def log_message(self, *a): pass Server(("127.0.0.1", 9003), H).serve_forever() EOF ``` #### 2 · Write the baseline Envoy config (no compression, listens on :10001) ```sh cat > envoy-baseline.yaml << 'EOF' static_resources: listeners: - name: ingress address: { socket_address: { address: 127.0.0.1, port_value: 10001 } } filter_chains: - filters: - name: envoy.filters.network.http_connection_manager typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager stat_prefix: ingress route_config: virtual_hosts: - name: all domains: ["*"] routes: - match: { prefix: "/" } route: { cluster: app } http_filters: - name: envoy.filters.http.router typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router clusters: - name: app type: STATIC connect_timeout: 1s load_assignment: cluster_name: app endpoints: - lb_endpoints: - endpoint: { address: { socket_address: { address: 127.0.0.1, port_value: 9003 } } } admin: address: { socket_address: { address: 127.0.0.1, port_value: 9901 } } EOF ``` #### 3 · Write the tuned config: same, plus a gzip compressor filter (listens on :10002) ```sh cat > envoy-tuned.yaml << 'EOF' static_resources: listeners: - name: ingress address: { socket_address: { address: 127.0.0.1, port_value: 10002 } } filter_chains: - filters: - name: envoy.filters.network.http_connection_manager typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager stat_prefix: ingress route_config: virtual_hosts: - name: all domains: ["*"] routes: - match: { prefix: "/" } route: { cluster: app } http_filters: - name: envoy.filters.http.compressor typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.compressor.v3.Compressor response_direction_config: common_config: min_content_length: 100 content_type: ["application/json"] compressor_library: name: gzip typed_config: "@type": type.googleapis.com/envoy.extensions.compression.gzip.compressor.v3.Gzip compression_level: BEST_SPEED - name: envoy.filters.http.router typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router clusters: - name: app type: STATIC connect_timeout: 1s load_assignment: cluster_name: app endpoints: - lb_endpoints: - endpoint: { address: { socket_address: { address: 127.0.0.1, port_value: 9003 } } } admin: address: { socket_address: { address: 127.0.0.1, port_value: 9902 } } EOF ``` #### 4 · Terminal 1: the backend ```sh python3 backend.py ``` #### 5 · Terminal 2: baseline Envoy ```sh envoy -c envoy-baseline.yaml --base-id 1 --log-level error ``` #### 6 · Terminal 3: tuned Envoy (a different --base-id lets two Envoys run side by side) ```sh envoy -c envoy-tuned.yaml --base-id 2 --log-level error ``` #### 7 · Terminal 4: what does a client receive? ```sh for p in 10001 10002; do echo "port $p: $(curl -s -H 'Accept-Encoding: gzip' http://127.0.0.1:$p/ | wc -c) bytes" done ``` Sample output: ``` port 10001: 258780 bytes port 10002: 23550 bytes ``` #### 8 · Terminal 4: let resonate validate the fix on both ```sh mkdir baseline tuned (cd baseline && resonate hit http://127.0.0.1:10001/ -H 'Accept-Encoding: gzip' \ --expect-header 'Content-Encoding: gzip' --rate 100 --duration 5s) (cd tuned && resonate hit http://127.0.0.1:10002/ -H 'Accept-Encoding: gzip' \ --expect-header 'Content-Encoding: gzip' --rate 100 --duration 5s) ``` Sample output: ``` Requests [total, rate] 599, 119.99/sec Duration [total] 4.992s Success [ratio] 0.00% Latencies [min, mean, p50, p75, p90, p95, p99, max] 0s, 0s, 0s, 0s, 0s, 0s, 0s, 0s Bytes [in, out] 155009220, 0 Errors: [599] check failed: header "Content-Encoding": got "", want "gzip" Requests [total, rate] 599, 119.80/sec Duration [total] 5s Success [ratio] 100.00% Latencies [min, mean, p50, p75, p90, p95, p99, max] 943µs, 4.716ms, 3.09ms, 3.222ms, 12.165ms, 17.203ms, 22.757ms, 24.215ms Bytes [in, out] 155009220, 0 Status Codes [code:count] 200:599 ``` #### 9 · Terminal 4: what does the optimization cost? Same load, no header check, then diff ```sh (cd baseline && resonate hit http://127.0.0.1:10001/ -H 'Accept-Encoding: gzip' --rate 200 --duration 10s) (cd tuned && resonate hit http://127.0.0.1:10002/ -H 'Accept-Encoding: gzip' --rate 200 --duration 10s) resonate compare baseline/report.json tuned/report.json ``` Sample output: ``` Metric Baseline Current Change requests 2201 2199 -0.1% rate 220.13 219.90 -0.1% success_rate 100.00% 100.00% +0.0% error_rate 0.00% 0.00% +0.0% latency_mean 4.054ms 4.782ms +18.0% latency_p50 936µs 2.509ms +167.9% latency_p75 1.054ms 3.076ms +191.9% latency_p90 1.552ms 3.416ms +120.1% latency_p95 36.93ms 25.461ms -31.1% latency_p99 38.83ms 40.829ms +5.1% latency_max 39.76ms 44.617ms +12.2% ``` Step 7 shows the payload shrinking from about 259 KB to about 24 KB. In step 8 the baseline fails every request because there is no Content-Encoding: gzip header, while the tuned Envoy passes 100%: that header check is how resonate proves the fix is really in place, and the same flag works as a CI guard against someone dropping the filter. Step 9 is the honest part: on loopback there is no bandwidth to save, so compression only adds CPU (the median was 2 to 3 times slower here) and the tail numbers move around from run to run. Resonate counts bytes after decompression, so use curl for wire size. Gzip pays off on real networks with large, compressible responses; always measure before and after. ### Do all events arrive? You run a streaming endpoint: live feeds, notifications or an LLM-style API. Does every stream deliver all its events when many are open at once? #### 1 · Write a tiny SSE server (Python stdlib): each stream sends 10 events, 20 ms apart ```sh cat > sse_server.py << 'EOF' import time from urllib.parse import urlparse, parse_qs from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler class Server(ThreadingHTTPServer): request_queue_size = 128 class H(BaseHTTPRequestHandler): protocol_version = "HTTP/1.1" def do_GET(self): n = int(parse_qs(urlparse(self.path).query).get("n", ["10"])[0]) # events per stream self.send_response(200) self.send_header("Content-Type", "text/event-stream") self.send_header("Connection", "close") self.end_headers() for i in range(n): self.wfile.write(f"id: {i}\ndata: tick {i}\n\n".encode()); self.wfile.flush() time.sleep(0.02) self.close_connection = True def log_message(self, *a): pass Server(("127.0.0.1", 8765), H).serve_forever() EOF ``` #### 2 · Terminal 1: start it ```sh python3 sse_server.py ``` #### 3 · Terminal 2: describe the test (one stream per iteration, 10 events each) ```sh cat > sse.yaml << 'EOF' protocol: sse load: duration: 15s rate: 20 workers: 20 sse: url: http://127.0.0.1:8765/events events: 10 # succeed once 10 events arrive... duration: 5s # ...or fail if they haven't within 5s timeout: 5s assertions: - metric: success_rate min: "0.99" EOF ``` #### 4 · Terminal 2: check one iteration, then run it ```sh resonate run sse.yaml --dry-run resonate run sse.yaml ``` Sample output: ``` Requests [total, rate] 3466, 231.05/sec Duration [total] 15.001s Success [ratio] 100.00% Latencies [min, mean, p50, p75, p90, p95, p99, max] 4µs, 19.334ms, 23.282ms, 25.002ms, 25.068ms, 25.1ms, 25.133ms, 25.4ms Bytes [in, out] 63000, 0 Status Codes [code:count] 200:3466 ``` #### 5 · Break it on purpose: a server that closes every stream after only 5 events ```sh sed 's#/events#/events?n=5#' sse.yaml > short.yaml resonate run short.yaml echo "exit code: $?" ``` Sample output: ``` Requests [total, rate] 2220, 147.99/sec Duration [total] 15.001s Success [ratio] 85.72% Latencies [min, mean, p50, p75, p90, p95, p99, max] 6µs, 15.789ms, 21.938ms, 24.674ms, 25.084ms, 25.117ms, 25.166ms, 26.748ms Bytes [in, out] 31700, 0 Status Codes [code:count] 200:1903 Errors: [317] stream ended after 5 of 10 events 1 of 1 assertion(s) failed: - success_rate: got 0.86, want >= 0.99 error: 1 assertion(s) failed exit code: 3 ``` Resonate reports the connection and every event separately, so the request count is about 11 per stream. Step 5 shows the failure that matters for streams: the server hangs up early. The Errors table says exactly why (stream ended after 5 of 10 events), the success-rate assertion fails, and the exit code is non-zero, ready for CI. The same scenario type works against LLM-style APIs that stream a token per event. ### Are your gRPC calls right? You have a gRPC service and no load tooling for it. Does your service hold up under load, and does it return the right status codes? #### 1 · Terminal 1: run the example gRPC server straight from GitHub (needs Go, no clone) ```sh go run github.com/jecklgamis/resonate/examples/testgrpcserver@main -log-requests=false ``` #### 2 · Terminal 2: describe the test (create a user, then read it back) ```sh cat > grpc.yaml << 'EOF' protocol: grpc load: duration: 15s rate: 100 workers: 20 grpc: address: 127.0.0.1:50051 plaintext: true # no TLS on the demo server timeout: 5s calls: - method: acme.user.v1.UserService/CreateUser message: '{"name":"{{randChoice "alice" "bob" "carol"}}","age":{{randInt 18 65}}}' extract: user_id: json:id - method: acme.user.v1.UserService/GetUser message: '{"id":"{{.Vars.user_id}}"}' expect_code: [OK] expect_body: json:name: "" # the field must exist assertions: - metric: success_rate min: "0.99" - metric: latency_p95 max: 50ms EOF ``` #### 3 · Terminal 2: check one iteration, then run it ```sh resonate run grpc.yaml --dry-run resonate run grpc.yaml ``` Sample output: ``` Requests [total, rate] 3198, 213.31/sec Duration [total] 14.992s Success [ratio] 100.00% Latencies [min, mean, p50, p75, p90, p95, p99, max] 103µs, 378µs, 328µs, 395µs, 484µs, 563µs, 2.4ms, 6.355ms Bytes [in, out] 103298, 29771 Status Codes [code:count] 200:3198 ``` #### 4 · Break it on purpose: expect OK for a user that does not exist ```sh cat > missing.yaml << 'EOF' protocol: grpc load: duration: 5s rate: 50 workers: 10 grpc: address: 127.0.0.1:50051 plaintext: true calls: - method: acme.user.v1.UserService/GetUser message: '{"id":"no-such-user"}' expect_code: [OK] # wrong on purpose: a missing user should be NOT_FOUND assertions: - metric: success_rate min: "0.99" EOF resonate run missing.yaml ``` Sample output: ``` Requests [total, rate] 299, 59.79/sec Duration [total] 5.001s Success [ratio] 0.00% Latencies [min, mean, p50, p75, p90, p95, p99, max] 0s, 0s, 0s, 0s, 0s, 0s, 0s, 0s Bytes [in, out] 0, 4186 Errors: [299] check failed: code: got NOT_FOUND (user "no-such-user" not found), want one of [OK] 1 of 1 assertion(s) failed: - success_rate: got 0.00, want >= 0.99 error: 1 assertion(s) failed ``` No .proto file is needed: resonate reads the schema from the server through reflection, and extract carries the id from CreateUser into GetUser. Each iteration is two calls, so requests per second is about double the iteration rate. In step 4 the server correctly answers NOT_FOUND, so expecting OK fails. Change expect_code to [NOT_FOUND] and the run passes: asserting an error path on purpose is a legitimate test too. ### Is your test lying? Your load test says everything is fast, but real users complain about stalls. Is your load test hiding the slow requests? #### 1 · Write a backend that freezes for 3 seconds in the middle of every run ```sh cat > stall.py << 'EOF' import time, threading from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler lock, t0, last = threading.Lock(), 0.0, 0.0 class Server(ThreadingHTTPServer): request_queue_size = 256 class H(BaseHTTPRequestHandler): protocol_version = "HTTP/1.1" def do_GET(self): global t0, last now = time.time() with lock: if now - last > 2: t0 = now # idle for 2s means a new run: restart the clock last = now elapsed = now - t0 if 4 <= elapsed < 7: # the stall: everything waits until second 7 time.sleep(7 - elapsed) time.sleep(0.005) # normal service time self.send_response(200); self.send_header("Content-Length", "2"); self.end_headers(); self.wfile.write(b"ok") def log_message(self, *a): pass Server(("127.0.0.1", 9100), H).serve_forever() EOF ``` #### 2 · Terminal 1: start it ```sh python3 stall.py ``` #### 3 · Terminal 2: closed model, 10 workers each waiting for their previous reply ```sh mkdir closed open (cd closed && resonate hit http://127.0.0.1:9100/ --workers 10 --duration 12s) ``` Sample output: ``` Requests [total, rate] 12374, 1031.14/sec Duration [total] 12s Success [ratio] 99.98% Latencies [min, mean, p50, p75, p90, p95, p99, max] 5.168ms, 8.896ms, 6.509ms, 6.767ms, 7.094ms, 7.311ms, 7.852ms, 3.008409s Bytes [in, out] 24742, 0 Status Codes [code:count] 200:12371 Errors: [3] Get "http://127.0.0.1:9100/": context deadline exceeded ``` #### 4 · Terminal 2: open model, 100 new requests per second no matter how slow the server is (wait 3s after step 3 so the server resets its clock) ```sh sleep 3 (cd open && resonate hit http://127.0.0.1:9100/ --rate 100 --max-workers 1000 --duration 12s) ``` Sample output: ``` Requests [total, rate] 1299, 108.28/sec Duration [total] 11.997s Success [ratio] 100.00% Latencies [min, mean, p50, p75, p90, p95, p99, max] 5.332ms, 355.828ms, 6.685ms, 16.54ms, 1.709179s, 2.359296s, 2.87939s, 3.017802s Bytes [in, out] 2598, 0 Status Codes [code:count] 200:1299 ``` #### 5 · Terminal 2: put them side by side ```sh resonate compare closed/report.json open/report.json ``` Sample output: ``` Metric Baseline Current Change requests 12374 1299 -89.5% rate 1031.14 108.28 -89.5% success_rate 99.98% 100.00% +0.0% error_rate 0.02% 0.00% -100.0% latency_mean 8.896ms 355.828ms +3899.7% latency_p50 6.509ms 6.685ms +2.7% latency_p75 6.767ms 16.54ms +144.4% latency_p90 7.094ms 1.709179s +23992.4% latency_p95 7.311ms 2.359296s +32168.9% latency_p99 7.852ms 2.87939s +36570.6% ``` Same server, same 3-second freeze, two very different answers. The closed model has only 10 requests in flight, so only 10 requests ever feel the freeze and p99 reads 7.9ms (only the max, 3s, gives it away). The open model keeps arriving on schedule like real users do, so everyone who shows up during the freeze waits, and p99 is 2.9s. A closed loop politely waits for a slow server and so under-samples exactly the requests you care about (coordinated omission). Use --rate with --max-workers when you want the latency your users would feel. The few context deadline exceeded errors in the closed run are requests cut off when the run ended. ### Did it get slower? You are about to ship a change and want CI to block it if performance regresses. Did your last change make the API slower? #### 1 · Write a tiny API whose handler time you can dial (stands in for your real code) ```sh cat > api.py << 'EOF' import os, time from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler DELAY = float(os.environ.get("DELAY_MS", "10")) / 1000 class Server(ThreadingHTTPServer): request_queue_size = 128 class H(BaseHTTPRequestHandler): protocol_version = "HTTP/1.1" def do_GET(self): time.sleep(DELAY) # stands in for your real handler's work self.send_response(200); self.send_header("Content-Length", "2") self.end_headers(); self.wfile.write(b"ok") def log_message(self, *a): pass Server(("127.0.0.1", 18081), H).serve_forever() EOF ``` #### 2 · Terminal 1: start version 1 ```sh DELAY_MS=10 python3 api.py ``` #### 3 · Terminal 2: record a baseline from version 1 ```sh mkdir baseline (cd baseline && resonate hit http://127.0.0.1:18081/ --rate 100 --duration 10s) ``` Sample output: ``` Requests [total, rate] 1098, 109.79/sec Duration [total] 10.001s Success [ratio] 100.00% Latencies [min, mean, p50, p75, p90, p95, p99, max] 10.22ms, 12.465ms, 12.82ms, 12.943ms, 13.115ms, 14.828ms, 15.802ms, 17.628ms Bytes [in, out] 2196, 0 Status Codes [code:count] 200:1098 ``` #### 4 · Terminal 1: ship a slower change (Ctrl-C the old one first) ```sh DELAY_MS=14 python3 api.py ``` #### 5 · Terminal 2: run the same load against the changed API ```sh mkdir candidate (cd candidate && resonate hit http://127.0.0.1:18081/ --rate 100 --duration 10s) ``` Sample output: ``` Requests [total, rate] 1099, 109.90/sec Duration [total] 10s Success [ratio] 99.91% Latencies [min, mean, p50, p75, p90, p95, p99, max] 14.346ms, 17.583ms, 17.859ms, 17.94ms, 18.039ms, 23.495ms, 25.362ms, 26.648ms Bytes [in, out] 2196, 0 Status Codes [code:count] 200:1098 Errors: [1] Get "http://127.0.0.1:18081/": context deadline exceeded ``` #### 6 · Terminal 2: compare to the baseline and fail if p95 got more than 20% worse ```sh resonate compare baseline/report.json candidate/report.json --fail-on 'latency_p95>20%' echo "exit code: $?" ``` Sample output: ``` Metric Baseline Current Change requests 1098 1099 +0.1% rate 109.79 109.90 +0.1% success_rate 100.00% 99.91% -0.1% error_rate 0.00% 0.09% n/a latency_mean 12.465ms 17.583ms +41.1% latency_p50 12.82ms 17.859ms +39.3% latency_p75 12.943ms 17.94ms +38.6% latency_p90 13.115ms 18.039ms +37.5% latency_p95 14.828ms 23.495ms +58.5% latency_p99 15.802ms 25.362ms +60.5% FAIL latency_p95>20%: 14.828ms -> 23.495ms (+58.5%) error: 1 of 1 regression rule(s) matched exit code: 1 ``` #### 7 · Terminal 1: roll the change back (Ctrl-C first) ```sh DELAY_MS=10 python3 api.py ``` #### 8 · Terminal 2: rerun and compare again ```sh mkdir rollback (cd rollback && resonate hit http://127.0.0.1:18081/ --rate 100 --duration 10s) resonate compare baseline/report.json rollback/report.json --fail-on 'latency_p95>20%' echo "exit code: $?" ``` Sample output: ``` Metric Baseline Current Change requests 1098 1098 +0.0% rate 109.79 109.79 +0.0% success_rate 100.00% 100.00% +0.0% error_rate 0.00% 0.00% +0.0% latency_mean 12.465ms 12.544ms +0.6% latency_p50 12.82ms 12.861ms +0.3% latency_p75 12.943ms 12.976ms +0.3% latency_p90 13.115ms 13.14ms +0.2% latency_p95 14.828ms 15.917ms +7.3% latency_p99 15.802ms 16.876ms +6.8% ok latency_p95>20%: 14.828ms -> 15.917ms (+7.3%) exit code: 0 ``` Every run writes a report.json, and compare turns two of them into a verdict: the rule latency_p95>20% means fail when p95 is more than 20% slower than the baseline. Here the 4 ms slowdown is +58% and fails with exit code 1; the rollback passes. Pick a threshold above your noise: an unchanged rerun of this very test drifted about 7% on p95, so a 10% rule would have flaked. In CI, keep the baseline report.json from your main branch, run compare in the pull-request job, and a non-zero exit fails the build. You can gate one step too, for example login:latency_p99>20ms. ### How much can it take? You need a capacity number before a launch and do not have one yet. How much traffic can your service take before p95 breaks? #### 1 · Write a service that can work on 4 requests at once, 10 ms each (about 400 req/s at best) ```sh cat > api.py << 'EOF' import threading, time from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler slots = threading.Semaphore(4) # the service can work on 4 requests at once class Server(ThreadingHTTPServer): request_queue_size = 512 class H(BaseHTTPRequestHandler): protocol_version = "HTTP/1.1" def do_GET(self): with slots: time.sleep(0.01) # 10 ms of work each: capacity is about 4 / 0.01 = 400 req/s self.send_response(200); self.send_header("Content-Length", "2") self.end_headers(); self.wfile.write(b"ok") def log_message(self, *a): pass Server(("127.0.0.1", 18082), H).serve_forever() EOF ``` #### 2 · Terminal 1: start it ```sh python3 api.py ``` #### 3 · Describe a staircase: hold each rate for 8 seconds, then step up ```sh cat > steps.yaml << 'EOF' protocol: http load: max_workers: 300 # let concurrency grow so the rate is held as latency rises stages: - { rate: 100 } - { duration: 8s, rate: 100 } - { rate: 200 } - { duration: 8s, rate: 200 } - { rate: 300 } - { duration: 8s, rate: 300 } - { rate: 400 } - { duration: 8s, rate: 400 } - { rate: 500 } - { duration: 8s, rate: 500 } - { rate: 600 } - { duration: 8s, rate: 600 } http: timeout: 5s targets: - method: GET url: http://127.0.0.1:18082/ EOF ``` #### 4 · A small script to read the run one step at a time ```sh cat > steps.py << 'EOF' import collections, datetime as dt, json rows = [json.loads(l) for l in open("results.jsonl")] t0 = min(dt.datetime.fromisoformat(r["timestamp"]) for r in rows) steps = collections.defaultdict(list) for r in rows: steps[int((dt.datetime.fromisoformat(r["timestamp"]) - t0).total_seconds() // 8)].append(r) print("step req/s errors p50 ms p95 ms") for k in sorted(steps): v = steps[k]; lat = sorted(x["latency_ms"] for x in v if x["latency_ms"] > 0) or [0] err = sum(1 for x in v if x.get("error") or x.get("status_code", 200) >= 400) print(f"{k + 1:>4} {len(v) / 8:>7.0f} {100 * err / len(v):>6.1f}% {lat[len(lat) // 2]:>8.1f} {lat[int(len(lat) * .95)]:>8.1f}") EOF ``` #### 5 · Terminal 2: run the staircase ```sh resonate run steps.yaml ``` Sample output: ``` Requests [total, rate] 13001, 270.84/sec Duration [total] 48.003s Success [ratio] 98.85% Latencies [min, mean, p50, p75, p90, p95, p99, max] 10.475ms, 501.521ms, 393.478ms, 893.911ms, 902.3ms, 1.757413s, 1.784676s, 3.541432s Bytes [in, out] 25702, 0 Status Codes [code:count] 200:12851 Errors: warning: target rate could not always be sustained — peak concurrency hit 300 in-flight iterations; raise --max-workers to open more headroom [error details trimmed] ``` #### 6 · Terminal 2: where did it bend? ```sh python3 steps.py ``` Sample output: ``` step req/s errors p50 ms p95 ms 1 100 0.0% 13.0 13.2 2 200 0.0% 13.0 13.3 3 300 0.0% 16.7 25.6 4 372 0.0% 686.0 1760.5 5 336 0.0% 894.1 1773.7 6 317 5.9% 891.9 1770.1 ``` The overall summary hides the story; the per-step table shows it. Up to 300 req/s the service answers in about 13 ms. At the 400 step latency jumps to seconds, and delivered throughput tops out near 370 and then falls, with errors appearing at the top step. That bend is the knee: past it, extra load only adds queueing. Quote a capacity below the knee, with headroom. The max_workers warning is expected, because the generator keeps opening concurrency to hold the target rate while the server slows down (an open-model test). Your numbers will differ; the method is the point. ### Is your limit working? You added a rate limit to protect an endpoint and want proof it really engages. Is your rate limit working, and is it still letting normal traffic through? #### 1 · Write a backend (brew install nginx for the proxy) ```sh cat > backend.py << 'EOF' import sys from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler port, name = int(sys.argv[1]), sys.argv[2].encode() class Server(ThreadingHTTPServer): request_queue_size = 128 # the default of 5 drops connections under load class H(BaseHTTPRequestHandler): protocol_version = "HTTP/1.1" def do_GET(self): self.send_response(200); self.send_header("Content-Length", str(len(name))) self.end_headers(); self.wfile.write(name) def log_message(self, *a): pass Server(("127.0.0.1", port), H).serve_forever() EOF ``` #### 2 · Write an nginx config that allows 50 requests/sec per client, with room for a burst of 30 ```sh cat > nginx.conf << 'EOF' events {} http { limit_req_zone $binary_remote_addr zone=perip:10m rate=50r/s; limit_req_log_level warn; # keep rejected requests out of the terminal server { listen 127.0.0.1:8890; location / { limit_req zone=perip burst=30 nodelay; limit_req_status 429; proxy_pass http://127.0.0.1:9001; } } } EOF ``` #### 3 · Terminal 1: the backend ```sh python3 backend.py 9001 A ``` #### 4 · Terminal 2: nginx, in the foreground ```sh nginx -c "$PWD/nginx.conf" -e /dev/stderr -g 'daemon off;' ``` #### 5 · Terminal 3: under the limit (25/s), where nothing should be rejected ```sh resonate hit http://127.0.0.1:8890/ --rate 25 --duration 10s \ --assert 'success_rate>=0.99' ``` Sample output: ``` Requests [total, rate] 274, 27.40/sec Duration [total] 10s Success [ratio] 100.00% Latencies [min, mean, p50, p75, p90, p95, p99, max] 412µs, 1.446ms, 1.171ms, 1.262ms, 2.078ms, 4.293ms, 5.71ms, 6.184ms Bytes [in, out] 274, 0 Status Codes [code:count] 200:274 ``` #### 6 · Terminal 3: over the limit (100/s), where the excess should get 429 ```sh resonate hit http://127.0.0.1:8890/ --rate 100 --duration 10s \ --assert 'success>=480' --assert 'success<=600' ``` Sample output: ``` Requests [total, rate] 1099, 109.89/sec Duration [total] 10.001s Success [ratio] 48.23% Latencies [min, mean, p50, p75, p90, p95, p99, max] 86µs, 1.194ms, 748µs, 1.096ms, 2.402ms, 5.292ms, 9.699ms, 13.327ms Bytes [in, out] 96691, 0 Status Codes [code:count] 200:530, 429:569 ``` Two runs prove both sides of a limit. Under it, every request succeeds. Over it, nginx answers 429 for the excess and accepts 50 per second plus the 30-request burst: 530 over 10 seconds, so both success-count assertions pass (exit code 0). A limit that does nothing would fail success<=600; one that is too strict would fail success_rate>=0.99 in step 5. A tip from testing this: with burst=10, 14 normal requests were rejected in the first millisecond, because resonate starts a run with a small flood. burst is what absorbs spikes like that.