Documentation
Overview
Stubsmith is a traffic-capture and test-fixture platform built around three stages: capture, mask, and serve. The SDK (Python or Node) intercepts outbound HTTP calls inside your own infrastructure, applies masking rules fail-closed, and posts the masked payload to the Stubsmith ingest endpoint (POST /v1/captures at https://ingest.stubsmith.dev). A raw request or response value crosses your network boundary only for a scalar you reviewed and gave an explicit keep rule; by default nothing has one. Two things fall outside that rule: a path segment the normalisation heuristic cannot recognise as an identifier, and the allowlisted transport headers a stub needs in order to be replayed. Both are covered under Path templates and Field rules below.
Capture runs through the Python and Node SDKs. The ingest endpoint accepts only payloads the SDK marks as masked. That mark is set by the client, so the guarantee holds for the official SDKs, not for a client you write yourself. Go and Java clients are planned. Every SDK example below has a Python and a Node tab. The HTTP fixtures API is language-agnostic.
The trust boundary is inside your process. The SDK replaces values with typed stubs before any network call is made. What the Stubsmith server receives is a masked body and a set of structural fingerprints (field names, key-paths, query-parameter names, and content type), not the original values, apart from any scalar you explicitly kept. The server accepts only payloads the SDK marks as masked, applies an image backstop, and classifies the traffic; it does not perform the primary masking.
Each captured request is classified into a request type, a (method, path pattern) pair. New fingerprint shapes are held in a pending state until an operator reviews and approves them in the Request Types editor. Review is over field names and paths, not over stored plaintext. Once a fingerprint is approved, subsequent captures matching it are stored as fixtures. The fixtures API (GET /v1/fixtures) returns every distinct response variant your app has seen for a given request type, ready to be parametrized over by your test suite. The Python SDK wraps this into a single function call.
Capturing traffic
Install the SDK
The SDKs are open source: stubsmith-python on PyPI and stubsmith-node on npm, both MIT. The Python SDK needs Python 3.9 or later, the Node SDK Node 22.12 or later.
pip install stubsmithimport stubsmith
# Add to your application entrypoint or conftest.py.
# The project key is read from STUBSMITH_API_KEY.
stubsmith.install(url="https://ingest.stubsmith.dev/v1/captures")
# Calls made with requests or httpx are now captured.npm install stubsmithimport { install } from "stubsmith";
// CommonJS: const { install } = require("stubsmith");
// Call before importing HTTP clients. The project key is read from STUBSMITH_API_KEY.
install({ url: "https://ingest.stubsmith.dev/v1/captures" });
// Calls made with node:http, node:https and global fetch are now captured,
// so axios, got and node-fetch are covered too.In Python, install() instruments whichever of requests and httpx your app already has. If you are starting without either, install the one you use with it: pip install "stubsmith[requests]" or pip install "stubsmith[httpx]". The testing extra adds responses, which only register and register_template need; the stubsmith.testing module is Python-only.
In Node, options are camelCase (apiKey, url, sampleRate). The environment variables are the same in both SDKs: STUBSMITH_API_KEY, STUBSMITH_API_URL, STUBSMITH_BUNDLE, STUBSMITH_DEBUG, STUBSMITH_MASK_SALT and STUBSMITH_FLUSH_TIMEOUT. The rest of this section describes the Python SDK; the Node SDK's README lists the Node specifics.
Calling stubsmith.install() patches requests.sessions.Session.request and Session.send, and, for httpx, Client.send and AsyncClient.send (whichever library is importable). The module-level helpers requests.get, requests.post and the rest are covered by that, because each one builds a Session and calls Session.request. Since SDK 0.2.0, a PreparedRequest built by hand and sent with Session.send() is also captured. Traffic sent through urllib, urllib3, http.client or aiohttp is not captured. The SDK applies your masking rules fail-closed, computes structural fingerprints, and forwards the masked payload asynchronously to the ingest endpoint. The call is fire-and-forget by design so that capture can never block or raise an exception in the caller. If you want to confirm that capture is working, fingerprints appear in the dashboard as soon as the first capture is received and approved. To surface send and sync failures on stderr while debugging, pass debug=True to StubSmith() or set the environment variable STUBSMITH_DEBUG=1 before your process starts. Either form activates the same path. This does not change the fire-and-forget contract; it only makes failures visible.
What install() connects to
The SDK reaches two hosts, which matters if you allow-list outbound traffic. It POSTs masked captures to https://ingest.stubsmith.dev/v1/captures, and it polls https://app.stubsmith.dev/api/v1/sdk/sync for your masking rules every 60 seconds by default (rules_poll_interval). Both are overridable with the url and backend_url arguments. It starts two daemon threads, stubsmith-sender and stubsmith-rules-cache. Those names are Python-level only: CPython does not set the operating system's thread name, so a native thread dump or /proc/<pid>/task/*/comm shows python for both. Read threading.enumerate() to see them, or better, call stubsmith.is_installed() to check that capture is armed.
If the rules API is unreachable, capture fails closed and degrades to full masking: the last known-good rules stay in effect, and a client that has never reached the API masks everything and keeps delivering captures. You get fully masked payloads marked novel, never silence and never cleartext. Use stubsmith.is_installed() to assert in your own code that capture is armed.
A rejected key is different. The SDK takes the project key from the api_key argument (apiKey in the Node SDK) or from the STUBSMITH_API_KEY environment variable. If the key is wrong or revoked, the ingest service refuses it as well as the rules API, so nothing is recorded. From Python SDK 0.9.1 and Node SDK 0.2.2, the first 401 or 403 logs one warning per process, without debug mode, and the warning never contains the key. Your application's requests and responses are unaffected.
Under a pre-fork server (gunicorn --preload, uWSGI, Celery, Odoo), call install() wherever is convenient: the client re-arms itself in each forked child, and repeated calls return the client already installed rather than starting more threads. Both properties require SDK version 0.2.0 or later; on 0.1.0, install once per worker instead.
Other languages
Capture is available today through the Python and Node SDKs. The ingest endpoint accepts only payloads marked sdk_masked: true, which the official SDKs set after masking in your process; a payload without that mark is rejected with 400 sdk_required, and nothing it sends is stored.
Ingest endpoint
The SDK posts to the ingest service at POST https://ingest.stubsmith.dev/v1/captures, authenticated with Authorization: Bearer <project-key>, or equivalently with the key in an X-Project-Token header. The ingest service is a separate Go process from the Node API that serves the dashboard and the fixtures endpoint.
Masking pipeline
Masking is fail-closed and happens at the SDK, inside your infrastructure, before any data is transmitted. The following describes the full pipeline from SDK to storage.
- SDK: primary masking (edge). The SDK applies your project's masking rules before posting to ingest. Every value is hidden unless a
keeprule explicitly permits it. By default every string, including an email address, becomes<masked>; numbers become0and booleans becomefalse. A stub email address is only produced when a mask rule for that field carriestype: "email"and theSTUBSMITH_MASK_SALTenvironment variable is set, in which case the SDK generates a format-preserving placeholder on your project's placeholder domain (defaultstub.invalid). A new field in a response is masked automatically until you configure a rule for it. The masking rules, both the per-fingerprintfield_rulesand the project-level global rule sets, are downloaded from the server and cached locally by the SDK. The server never applies these rules itself; it only provides them to the SDK. - Ingest: checks. The ingest service accepts only payloads marked
sdk_masked=true, which the official SDKs set after masking. Payloads lacking this flag are rejected with a400 sdk_requirederror. Ingest does not scan captures for personal data: masking happens in the SDK, and a server-side scan could only recognise an arbitrary subset of formats. On accepted payloads the server runs one safety net, clearly secondary to the masking layer:- Image backstop. Request and response bodies with an
image/*content type that do not match the canonical 1×1 pixel placeholder for their subtype are replaced with that placeholder. This enforces the same image-placeholder invariant the SDK applies.
- Image backstop. Request and response bodies with an
- Fingerprint pending gate. Every capture is classified into a fingerprint, the structural identity of its request shape. New fingerprint shapes are held in a pending state. While a fingerprint is pending, occurrence counts and response-variant metadata are recorded, but no capture bodies are stored. An operator opens the fingerprint in the Request Types editor, reviews the field names and paths (not stored plaintext), and saves a set of
field_rulesto approve it. Only after approval does the ingest service begin writing matching capture bodies to permanent storage.
What a capture looks like
The payload below is what the SDK actually posts for GET /v1/orders/1042?limit=50&include=customer returning a body with a name, an email, an IBAN and an amount, on an endpoint whose fingerprint has not been approved yet. That is the worst case for disclosure, because no keep rule exists to let anything through.
{
"sdk_masked": true,
"sdk_rule_version": "0",
"domain": "api.example.com",
"path_template": "/v1/orders/{id}",
"path": "/v1/orders/{id}?limit=%3Cmasked%3E&include=%3Cmasked%3E",
"method": "GET",
"status": 200,
"key_paths": [],
"resp_key_paths": ["id", "customer", "customer.name", "customer.email", "iban", "amount", "paid"],
"query_names": ["limit", "include"],
"req_header_names": ["accept", "accept-encoding", "connection", "user-agent", "x-request-id"],
"headers": { "Accept": "*/*", "X-Request-Id": "<masked>" },
"req_body": "<masked>",
"resp_body": "{\"id\": \"<masked>\", \"customer\": {\"name\": \"<masked>\", \"email\": \"<masked>\"}, \"iban\": \"<masked>\", \"amount\": \"<masked>\", \"paid\": false}",
"novel": true,
"resp_value_types": { "id": "uuid", "customer.email": "email", "iban": "iban", "amount": "decimal_amount" }
}No original value appears in this capture, including in the URL: the path is templated, so 1042 becomes {id}, and query values are masked while their names survive. The one thing here that is literal is "Accept": "*/*", an allowlisted transport header a stub needs in order to be replayed. key_paths and resp_key_paths are names only, and they are what you review in the Request Types editor. resp_value_types reports recognizable formats, not content: "iban" means the string parsed as an IBAN, and character composition is never inspected. novel: true with sdk_rule_version: "0" means no approved rules were in effect, which is why every value is masked; after you approve the fingerprint, scalars with a keep rule appear here verbatim, and that review step is the only way a real value ever reaches Stubsmith deliberately. The path is the one place an unrecognised identifier can arrive without one; see Path templates below.
Field rules
Each approved fingerprint has an associated set of field_rules: an array of objects with a path (namespaced dot-path), an action (mask or keep), and an optional semantic type hint (only valid on mask rules). Path namespaces that mask or keep data are body., query., header., resp., resp_header., and status-scoped forms such as resp:200.amount and resp_header:200.x-request-id.
path. is also accepted by the validator but is metadata only; the SDK does not apply rules under it yet.
The SDK downloads these rules via the sync endpoint and applies them fail-closed at capture time.
Important: keep rules apply to exact scalar paths only. A keep rule on a parent path does not keep nested scalars. Every scalar you want to preserve must have its own explicit keep rule. For example, a keep rule on body.customer does nothing for body.customer.id or body.customer.name, those scalars are still masked unless each has an explicit keep rule. A user who writes a keep rule on a parent object expecting nested fields to survive will silently lose those values.
[
{ "path": "body.customer.id", "action": "keep" },
{ "path": "body.customer.status", "action": "keep" },
{ "path": "body.amount", "action": "mask", "type": "decimal_amount" }
][
{ "path": "body.customer", "action": "keep" }
// body.customer.id and body.customer.status are still masked
]Keeping the fields your tests branch on
Masking is fail-closed, so the values a test asserts on are gone unless you keep them. A masked status is <masked>, a masked livemode is false, and a masked amount is 0, which means a declined charge and a successful one differ only in their HTTP status. That is the intended default, not a gap: the way to get a useful stub is to name the handful of scalars your tests read and keep each of them.
Low-cardinality fields cannot be protected by masking in any case. A field whose value comes from a short list is recoverable from its placeholder by trying every candidate, so for currency_code, country_code and any boolean the SDK refuses to generate a shape-preserving placeholder and emits the constant instead. Other enums, a status or an error code, will take a placeholder, but a synthetic string in place of card_declined tells a test nothing. Keep them explicitly rather than masking them:
[
{ "path": "resp:402.error.code", "action": "keep" },
{ "path": "resp:402.error.type", "action": "keep" },
{ "path": "resp.status", "action": "keep" },
{ "path": "resp.currency", "action": "keep" },
{ "path": "resp.livemode", "action": "keep" },
{ "path": "resp.amount", "action": "mask", "type": "decimal_amount" },
{ "path": "resp.customer.email", "action": "mask", "type": "email" }
]A kept value arrives as it was sent, so keep only fields whose values are not personal data: enum members, status codes, flags, currency and country codes, and identifiers you are comfortable storing. Review the field name in the Request Types editor before you keep it; the editor shows names and paths, never stored values.
Shape-preserving placeholders
By default a masked value is a constant of the right type: <masked> for a string, 0 for a number, false for a boolean, and null stays null. That is safe, but 0 fails a validator that expects a positive quantity and <masked> fails one that expects a timestamp.
Set STUBSMITH_MASK_SALT in the environment of the process running the SDK, and give the mask rule a semantic type, and the placeholder keeps the format of the real value instead. The vocabulary is email, uuid, iso8601, e164, iban, url, decimal_amount, integer_id, opaque_token and free_text. A masked UUID becomes a parseable version-4 UUID, a masked timestamp becomes a valid ISO 8601 instant, a masked IBAN carries a correct mod-97 checksum, and a masked amount is a non-zero decimal of the same JSON type as the original.
Placeholders are deterministic: the value is derived from a keyed blake2b hash of the real value, so the same input always produces the same placeholder and two fields that held the same production value still match after masking. That keeps joins and referential checks meaningful across a capture. It also means the salt must stay stable: change it and every placeholder changes with it. The salt never leaves your environment; Stubsmith neither receives nor stores it.
The type is a hint, not a mandate. The server infers it from the field's leaf name and has never seen a value, so it can be wrong: a field called charge_id may hold a string or an integer. Before generating, the SDK checks the hint against the value's actual runtime type and falls back to the constant placeholder on a mismatch, so a placeholder never changes the JSON type of a field. currency_code and country_code are always refused for the reason above, and a boolean always becomes false. For those, use keep.
Global anonymizer rule sets
A project can have any number of named global rule sets. Each rule set is independently toggled on or off. Enabled rule sets are merged and downloaded by the SDK as part of the sync response, and the SDK applies them at the edge as an additional masking pass on top of per-fingerprint field_rules. Manage rule sets in the Global Anonymizer section of the dashboard.
Rule sets support two mechanisms:
Field masks: field-name strings (e.g. password, authorization). Any field whose name matches, regardless of its depth in the body, is masked by the SDK.
Regex masks: pattern+replacement pairs applied by the SDK against the serialized body, useful for catching values that shift between fields across requests.
Fingerprints
A fingerprint is one unique traffic shape on one endpoint. Its key is the request method and path template plus a blake2b-64 digest over the request’s sorted key-paths, query-parameter names and content type. It records structure and names only. By default no values enter the digest; a variant key you opt in adds that value’s hash.
A variant key must be a field you keep: every approved fingerprint of the endpoint has a keep rule for it, and no project field mask names it. Its value is then stored in captures and visible in Review (project regex masks still apply to it). The API refuses a variant key on a field that is not kept with HTTP 422, and refuses a rule change or field mask that would mask a field in use as a variant key, naming the fingerprints that use it. Remove the variant key first if you want to mask that field. The SDK hashes a variant key on every request to the endpoint, so a new request shape is hashed before you review it, and a value a project regex mask replaces still adds its hash.
The fingerprint is computed by the SDK and sent to the ingest service. The server stores the fingerprint's structural metadata (key-paths, header names, query-parameter names, value types) and uses it to deduplicate traffic and drive the pending/approved state machine. Fingerprints are capped per plan: 50 on Free, 500 on Solo, 5,000 on Team, and 25,000 on Business. When the cap is reached, a capture with a new shape is not stored; ingest answers with HTTP 200 and a body of {"ok": false, "error": "fingerprint_limit_reached"}. Existing shapes keep capturing normally.
URL paths
/v1/charges/90421 and /v1/charges/90422 are the same endpoint, so the SDK turns the path into a template before it fingerprints anything. This happens in your process: a segment that becomes {id} is never sent.
Automatic ID grouping
Each segment is judged on its own. A segment becomes {id} when it is entirely digits, shaped like a UUID, or 16 or more hexadecimal characters. Every other segment is sent as written: /users/jane-doe stays /users/jane-doe, and a prefixed identifier like ch_3Nk9Qb does not collapse on its own.
This is the project setting Group IDs in paths automatically, on by default. Some APIs use numbers as route words, so /reports/2024 becomes /reports/{id} and every year lands in one request type. If that is wrong for your API, an org admin can turn the setting off on the Request Types page. With it off, only your own patterns group paths, and every other path, including /orders/123, is sent as written. Both SDKs and replay read the setting from your project, so capture and replay always agree. Running SDKs pick up a change at their next rules sync, within 60 seconds by default.
Everything else is sent as written
A path segment is part of the request line, not a body value, so it is not masked. An endpoint shaped like /users/ana@example.com sends the address to Stubsmith, and nothing on the server catches it, because Stubsmith does not scan captures for personal data. Keep personal data out of paths, or set a pattern for every route that carries it.
Setting a pattern
Open the request type in the dashboard and set its pattern, for example /users/{username}/orders. A segment written as {name} is a wildcard. A pattern takes precedence over automatic grouping. When several patterns match a path, only those with the same number of segments count, and the one with the most literal segments wins, ties broken alphabetically.
- Running SDKs pick up the pattern at their next rules sync, within 60 seconds by default. Until then, those requests arrive with the segment as written, and are stored under the pattern.
- The request types it matches are folded into one, and the raw values in its wildcard segments are removed from Stubsmith's database.
- A kept body field or header that contains the URL, such as a
Locationheader, still holds it. A pattern covers the request path only. - Database backups keep the old values until they expire, after 3 days.
Pattern suggestions
Automatic grouping cannot recognise slugs or short codes. When many request types differ only in one path segment (20 by default), the Request Types page suggests a pattern with that segment as a wildcard, and shows how many request types it folds and a few example values. The suggestion comes from the shape of your traffic, never from what a value looks like. An org admin can accept it, which sets the pattern as above, or dismiss it, so the same pattern is not suggested again. Nothing changes until you accept. The values that produced the suggestion have already reached Stubsmith; accepting removes them.
One more consequence worth planning for: the digest covers the request's key-paths and query-parameter names, so an optional field that is present in one request and absent in the next produces two fingerprints. An endpoint with five optional filters can produce many shapes, which is worth knowing on Free, where the cap is 50. If you are near the cap, approve the shapes you care about and check the fingerprint list for near-duplicates that differ only by an optional parameter.
Sample windows
There is no single project-wide capture window. Every fingerprint keeps its own rolling window of masked samples, and within a fingerprint every response status keeps its own window too. The key is (fingerprint, response status): a shape’s 200 and its 429 are separate windows that never evict each other.
This is deliberate. Under one shared budget, an endpoint whose 200 fires ten thousand times a day would push out the 429 it returned once, which is exactly the response a test most wants and the one you cannot reproduce on demand. Scoping the window per status keeps rare responses alongside common ones, and scoping it per fingerprint keeps a chatty endpoint from crowding out a quiet one.
The window is a count, not an age. When a new sample arrives for a (fingerprint, status) pair that is already full, the oldest sample in that pair is deleted, together with its request and response bodies. Nothing expires on a clock: an endpoint you exercise once a quarter keeps its samples until newer ones arrive, so a replay bundle built from it stays reproducible. The one exception is a masking rule change: samples captured under the old rule are deleted within about ten minutes. A high-traffic endpoint never stores more than its cap. Pruning runs inside the capture transaction on the ingest write path, so stored samples never exceed the cap rather than converging on it once a day.
The window size is set by your plan: 3 samples per response status on Free, 5 on Solo, 10 on Team, 25 on Business. Only approved fingerprints store samples at all. A pending fingerprint records its occurrence count and the response statuses it has returned, but no bodies, so its windows start filling from the moment you approve it.
Fingerprints, request types and generated stubs are not part of any window. They are your durable output and are kept until you delete them. To pull a window into your tests rather than only the latest sample, see getting the window into a bundle and looping every recorded response.
Using fixtures in tests
GET /v1/fixtures is the primary consumer endpoint. It returns all recorded response variants for a given request type, one per unique status code when distinct=status is set, so your parametrized tests cover every real behaviour the API has exhibited.
Query parameters
| Parameter | Required | Description |
|---|---|---|
requestTypeId | One of | UUID of a configured request type. Mutually exclusive with method+path. |
method + path | One of | HTTP method and path. Resolved against configured request types, exact (non-dynamic) request type first, then dynamic pattern match; falls back to an exact path match on captures if no request type matches. |
distinct | No | Pass status to return one fixture per unique HTTP status code. Currently the only supported value. |
limit | No | Maximum fixtures to return (default 20, max 100). Selection scans the most recent 500 captures for the given method. |
curl example
# Requires env vars: STUBSMITH_API_URL, STUBSMITH_API_KEY
curl -G "${STUBSMITH_API_URL}/v1/fixtures" \
--data-urlencode "method=POST" \
--data-urlencode "path=/v1/charges/{id}" \
--data-urlencode "distinct=status" \
-H "Authorization: Bearer ${STUBSMITH_API_KEY}"Example response
{
"ok": true,
"request_type": {
"id": "rt_01j9x2...",
"method": "POST",
"path_pattern": "/v1/charges/{id}",
"is_dynamic": true
},
"count": 2,
"fixtures": [
{
"id": "cap_01j9...",
"captured_at": "2025-11-12T10:22:04Z",
"method": "POST",
"path": "/v1/charges/ch_abc123",
"status": 200,
"duration_ms": 84,
"request": {
"headers": { "content-type": "application/json" },
"body": "{\"amount\":0,\"currency\":\"usd\",\"customer\":\"<masked>\"}"
},
"response": {
"headers": { "content-type": "application/json" },
"body": "{\"id\":\"ch_abc123\",\"status\":\"succeeded\",\"amount\":4200}"
}
},
{
"id": "cap_01j8...",
"captured_at": "2025-11-10T08:14:51Z",
"method": "POST",
"path": "/v1/charges/ch_def456",
"status": 402,
"duration_ms": 61,
"request": {
"headers": { "content-type": "application/json" },
"body": "{\"amount\":0,\"currency\":\"usd\",\"customer\":\"<masked>\"}"
},
"response": {
"headers": { "content-type": "application/json" },
"body": "{\"error\":{\"code\":\"card_declined\",\"message\":\"Your card was declined.\"}}"
}
}
]
}request.body and response.body are each a JSON string holding the masked body as it was captured, not a parsed object.
Offline replay
stubsmith.replay() answers your code's outbound HTTP calls from recordings. It intercepts requests and, from Python SDK 0.9.0, httpx (Client and AsyncClient); calls made through other libraries are not intercepted and attempt real network connections. Inside the block nothing sent through requests or httpx reaches the network and the dependency does not need to be running, so a test suite needs no API key once the bundle is on disk. A response captured through httpx client.stream() is stored without a body, so it replays with an empty body. Upgrading from 0.8: httpx calls inside replay() now get a recorded answer or StubNotFound instead of reaching the real service.
import stubsmith
def test_charge_is_declined():
with stubsmith.replay():
with pytest.raises(CardDeclined):
PaymentClient().charge(amount_cents=950_000, currency="USD")import { test } from "node:test";
import assert from "node:assert/strict";
import { replay } from "stubsmith";
test("charge is declined", async () => {
const r = replay();
try {
await assert.rejects(
new PaymentClient().charge({ amountCents: 950_000, currency: "USD" }),
CardDeclined,
);
} finally {
r.restore();
}
});On Node 24 you can write using r = replay(); instead of try/finally; it restores at the end of the block.
The bundle is found without configuration: an explicit path or dict, then $STUBSMITH_BUNDLE, then an upward search for .stubsmith/bundle.json. A request that matches no recording raises StubNotFound with a diff against the closest one; it never falls through to the network.
Looping every recorded response
replay() serves one response per request shape: the newest recording of the status that occurred most often. Everything else Stubsmith holds for that shape goes untested, which is usually where the bugs are, since the 429 and the 500 your API really returned are recorded and never exercised. replay_all() runs the block once per recording. Requires SDK 0.3.0 or later.
def test_parses_every_recorded_success():
for attempt in stubsmith.replay_all(statuses={200}):
with attempt:
result = connector.sync_orders()
assert result.orders is not None
def test_degrades_on_every_recorded_failure():
for attempt in stubsmith.replay_all(statuses={429, 500}):
with attempt:
with pytest.raises((RateLimited, UpstreamUnavailable)):
connector.sync_orders()import { replayAll } from "stubsmith";
test("parses every recorded success", async () => {
for await (const pass of replayAll({ statuses: [200] })) {
const result = await connector.syncOrders();
assert.notEqual(result.orders, undefined);
}
});
test("degrades on every recorded failure", async () => {
for await (const pass of replayAll({ statuses: [429, 500] })) {
await assert.rejects(connector.syncOrders());
}
});In Node each pass restores itself, so there is no inner block as in Python.
Filter by status unless you have a reason not to. It decides what a failing build means. Unfiltered, the loop feeds successes and failures into one test body, so the body has to hold for both; the day a 500 first appears in the window, assertions written when only 200s existed start failing, and the build breaks because the recording changed rather than because the code did. That is the opposite of what a failing test should tell you.
Filtered, every pass is the same kind of response, the assertions can be specific, and a failure means your code cannot handle a response your API genuinely returns. The filter also bounds the work: unfiltered, the pass count is roughly (samples per response) × (distinct statuses), and it grows with traffic on its own. A shape with recordings but none of the requested status raises StubNotFound naming the filter and listing what it does have, rather than serving a response the loop was told to exclude. Requires SDK 0.4.0 or later.
No endpoint is named. Which shapes a pass touches is discovered by running your code, so this works for a client whose call sequence you would rather not spell out, and for one whose sequence changes between passes because it branches on the response it got: a shape first reached on a later pass is still looped from its own first recording.
The first pass serves exactly what replay() serves, so a test that passes under replay() still passes on pass one. Iteration stops once every shape that was actually touched has served its last recording, and a shape with a shorter window keeps serving its final recording rather than raising, so one endpoint's thin history cannot truncate the loop for the rest.
Note what this asks of the test body: it runs against a 200 on one pass and possibly a 500 on the next, so the assertions have to hold across the whole recorded range. That is the point, and it is also why replay() remains the right tool for a test written against one known response. To pin a single response, pass select:
def test_backs_off_when_rate_limited():
with stubsmith.replay(select=stubsmith.by_status(429)):
with pytest.raises(RateLimited):
connector.sync_orders()import { replay, byStatus } from "stubsmith";
test("backs off when rate limited", async () => {
const r = replay({ select: byStatus(429) });
try {
await assert.rejects(connector.syncOrders(), RateLimited);
} finally {
r.restore();
}
});by_status raises StubNotFound when no recording of that status exists rather than quietly serving a different one, because a rate-limit test that ran against a 200 would pass while testing nothing.
Either way, served() reports what actually ran, so a test can assert its coverage instead of assuming it. Each entry carries domain, method, path_template, fingerprint, status, capture_id, captured_at, and its position in that shape's window as index and total.
with stubsmith.replay() as r:
connector.sync_orders()
assert {s.status for s in r.served()} == {200}const r = replay();
try {
await connector.syncOrders();
assert.deepEqual(new Set(r.served().map((s) => s.status)), new Set([200]));
} finally {
r.restore();
}served() lists every call in order. Field names are camelCase in Node (pathTemplate, capturedAt).
Getting the window into a bundle
A bundle holds one recording per response by default, so replay_all() would give one pass per status. Ask for the window explicitly:
export STUBSMITH_API_KEY=<your project key>
stubsmith pull --samples all \
--endpoint "GET /admin/orders" \
--endpoint "GET /admin/products" \
--out tests/bundles/admin.jsonexport STUBSMITH_API_KEY=<your project key>
npx stubsmith pull --samples all \
--endpoint "GET /admin/orders" \
--endpoint "GET /admin/products" \
--out tests/bundles/admin.jsonThe flags are the same in both SDKs. In Node, pullBundle() does the same from code without writing a file.
--samples above 1 requires --endpoint, because the full window for a whole project is not served. Repeat the flag to cover several endpoints in one file: that is one request each, merged for you, with duplicate request shapes dropped and any truncated report carried through. Repeating requires SDK 0.4.0 or later.
Scoping is what you want anyway: a project-wide bundle is capped at 2,000 request shapes (counted per endpoint and fingerprint) and reports the rest in truncated rather than carrying them. Commit the files and the suite needs no key and no network; treat a refresh like a lockfile change.
Or fetch at collection time, with no file involved:
BUNDLE = stubsmith.fetch_bundle("GET /admin/orders", samples="all")
def test_every_recorded_response():
for attempt in stubsmith.replay_all(BUNDLE):
with attempt:
connector.list_orders()import { fetchBundle, replayAll } from "stubsmith";
const bundle = await fetchBundle("GET /admin/orders", { samples: "all" });
test("every recorded response", async () => {
for await (const pass of replayAll({ bundle })) {
await connector.listOrders();
}
});fetchBundle() is async in Node, so it needs await.
Fetch once at module level, not per test: it is a blocking HTTP call. It needs a live key wherever the tests run, and recordings roll as the sample window rolls, so a failure you see today may not reproduce next week. Fetch live while iterating locally; commit a pulled bundle for CI.
That split matters more under replay_all() than under replay(). With a committed bundle the loop runs the same passes until someone re-pulls, and a new recording arrives as a reviewable file change. Fetching live, both the number of passes and the responses in them change as traffic changes, so a build can turn red with no commit behind it.
Replay bundles
GET /v1/replay/bundle returns everything needed to serve recorded responses with no network access, grouped by endpoint, then request shape, then response. It is what stubsmith pull writes to .stubsmith/bundle.json and what stubsmith.replay() reads. Authenticate with a project key.
Scope it to one endpoint with requestTypeId, or with method and path together. The path may be the concrete one your code calls (/admin/orders/1042); the server matches it against your recorded path patterns and resolves the request type itself.
# The whole project
curl -G "https://app.stubsmith.dev/api/v1/replay/bundle" \
-H "Authorization: Bearer ${STUBSMITH_API_KEY}"
# One endpoint, every recorded sample of every response
curl -G "https://app.stubsmith.dev/api/v1/replay/bundle" \
--data-urlencode "method=GET" \
--data-urlencode "path=/admin/orders" \
--data-urlencode "samples=all" \
-H "Authorization: Bearer ${STUBSMITH_API_KEY}"By default each response carries a single body: the most recent recording. That is all that is needed to serve a response, and it keeps a whole-project bundle small. Pass samples=N or samples=all and every response also carries a samples array holding its rolling window, newest first, each entry with its own capture_id, captured_at, headers and body. Use it to run a test against every recorded response for a shape rather than only the latest, which is how you find out your code breaks on the 500 your API returned last week.
samples above 1 requires a scope: fetching every sample for an entire project is rejected with a 400. The ceiling is 25, matching the largest plan's per-response window, and a larger request is clamped and reported in the response's truncated object rather than trimmed silently.
Always check truncated. A bundle is capped at 2,000 request shapes, 10 responses per shape, and 100 KB per body, and anything dropped is named there. A project larger than those caps needs per-endpoint bundles rather than one project-wide fetch.
If a recorded body cannot be read from storage, /v1/replay/bundle and /v1/fixtures fail the whole request rather than return a response without its body: 503 with a Retry-After header when storage is temporarily unreachable, so retry after that many seconds, and 500 when the body is gone. Both answer {"error":"body_unavailable"}. A response that genuinely had no body still comes back with body: null. stubsmith pull exits non-zero on either and leaves an existing bundle file untouched.
Fixtures in your test runner
Set STUBSMITH_API_URL and STUBSMITH_API_KEY before running tests, or pass them as keyword arguments to stubsmith.fixtures()(options to fixtures() in Node). In Python the call is a blocking fetch at pytest collection time; in Node it is async, and the same code runs under node:test, jest or vitest with their own test.
import pytest
import stubsmith
fixtures = stubsmith.fixtures("POST /v1/charges/{id}", distinct="status")
@pytest.mark.parametrize("fx", fixtures, ids=lambda f: str(f.status))
def test_handles_every_recorded_response(fx):
body = fx.response.json()
# assert your application contract against body and fx.status
...import { test } from "node:test";
import { fixtures } from "stubsmith";
const recorded = await fixtures("POST /v1/charges/{id}", { distinct: "status" });
for (const fx of recorded) {
test(`handles a recorded ${fx.status}`, () => {
const body = fx.response.json();
// assert your application contract against body and fx.status
});
}This fetches at collection time, so it needs a project key wherever the tests run, and it stubs nothing: you assert against the fixture yourself. To have your real client code answered from recordings with no key and no network, see offline replay above, which also covers looping every recording rather than one per status.
Without distinct="status" the call returns individual captures, newest first, capped at 100 and defaulting to 20. A test that parametrizes over many samples without raising limit is silently working from the first 20.
Pattern resolution note: passing a path containing {param} segments resolves only against configured request types that have a matching pattern. Passing a concrete path (e.g. /v1/charges/ch_abc123) is resolved against an exact (non-dynamic) request type first, then against a pattern-based request type via segment matching, falling back to an exact path match on captures if no request type is configured.
API reference
Stubsmith exposes two separate HTTP services. The ingest endpoint (POST /v1/captures) is served by the Go ingest service at https://ingest.stubsmith.dev. All other endpoints are served by the Node API at the app hostname. Both authenticate with Authorization: Bearer <project-key>; the ingest service also accepts the key in an X-Project-Token header. Dashboard operations use the session the dashboard signs in with, an HttpOnly cookie set by POST /v1/auth/login; it is not exposed to scripts or returned in the response. Where the table says JWT, the caller must hold the organisation admin role; other members receive 403.
| Method | Path | Auth | Purpose |
|---|---|---|---|
| POST | /v1/captures | Project key | Ingest a captured request/response (ingest service: https://ingest.stubsmith.dev) |
| GET | /v1/captures | JWT or project key | List captures for a project (newest first, max 100) |
| GET | /v1/captures/:id | JWT or project key | Full capture detail; append ?includeBodies=true to resolve offloaded body references |
| GET | /v1/request-types | JWT or project key | List request types (method, path, pattern, and pending/approved/rejected fingerprint counts) for a project |
| GET | /v1/request-types/:id | JWT or project key | Full detail including fingerprints, response variants, and field rules |
| PUT | /v1/request-types/:id/pattern | JWT or project key | Set a path pattern; absorbs matching concrete request type rows |
| GET | /v1/projects/:id/path-suggestions | JWT (org admin) | Suggested path patterns for request types that differ in one segment, with the number each would fold and example values. Accept one with the pattern route above on its request_type_id |
| POST | /v1/projects/:id/path-suggestions/dismiss | JWT (org admin) | Dismiss a suggestion permanently; body {domain, method, pattern} |
| DELETE | /v1/request-types/:id | JWT (org admin) or project key | Delete a request type along with its captures and stored bodies. A project key alone is sufficient, so treat project keys as capable of deleting stored fixtures |
| POST | /v1/review/:fingerprint_id/decision | Org-admin JWT or org API key with review:approve scope | Approve or reject a pending fingerprint; submit field_rules on approve. The pushed_capture field has been removed: a request that includes it is refused with 400. Project keys cannot review |
| POST | /v1/review/:fingerprint_id/reopen | Org-admin JWT or org API key with review:approve scope | Send an approved or rejected fingerprint back to pending |
| GET | /v1/review/queue | Org-admin JWT or org API key with review:read scope | List pending fingerprints awaiting review |
| PUT | /v1/request-types/:id/fingerprints/:fingerprintId/rules | Org-admin JWT | Update field_rules on an already-approved fingerprint; project keys are explicitly excluded. Editing masking rules is a privileged privacy action |
| GET | /v1/anonymizer/rules | JWT or project key | List named global anonymizer rule sets for a project |
| POST | /v1/anonymizer/rules | Org-admin JWT | Create a new named rule set; 409 if name already exists |
| PUT | /v1/anonymizer/rules/:id | Org-admin JWT | Partially update a rule set, accepts any combination of name, rules, and enabled |
| DELETE | /v1/anonymizer/rules/:id | Org-admin JWT | Delete a rule set. Project keys get 403 on all three rule-set writes; they keep read access for the sync |
| GET | /v1/sdk/sync | Project key | SDK rule sync: returns field_rules delta since cursor, path templates, merged global rule sets, per-request-type value-path config, and the project's email placeholder domain |
| GET | /v1/rules | JWT or project key | All approved field rules for a project in one list |
| GET | /v1/alerts | JWT or project key | List capture alerts |
| GET | /v1/fixtures | JWT or project key | Fetch fixture variants for a request type (see above) |
| GET | /v1/replay/bundle | Project key | Fetch the offline replay bundle used by stubsmith pull and stubsmith.replay() (see Replay bundles) |