← Back to the QA results

Net Atelier — Stripe QA test matrix: findings

• Date: 2026-09-15
• Target: https://next.net-atelier.com (QA, wired to qa.net-atelier.com)
• Method: Playwright + Chromium, driving the real signup → payment flow
• Runs from: Sydney 206.245.167.24, Dallas 79.141.173.27

Why this document has gaps in it

The matrix is 18 cases. Written up honestly, some of it is proven, some of it is blocked, and none of it is guessed. The blocker is not our tooling — it is a rate limit on the QA signup endpoint, documented below. Cases we could not reach are listed as UNTESTED rather than quietly dropped.

The headline result: QA can reach Stripe

The open question going in was whether QA has working Stripe credentials. It does.

POST /api/subscribe returned HTTP 200 with:

{"requires_action": true, "setup_intent_client_secret": "seti_..._secret_..."}

That is a real SetupIntent created against Stripe, correctly asking for authentication on a 3DS card. So the QA environment's Stripe integration is live and functioning, and the card matrix is testable in principle.

Blocker: the QA signup endpoint is rate limited

POST /api/auth/signup  ->  HTTP 429
{"error": {"message": "Too Many Attempts.", "status_code": 429,
  "debug": {"file": "/app/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php"}}}

What we measured:

Observation	Detail
Still 429 after	300 seconds of retries, 6 attempts
Still 429 after	>60 minutes on the Sydney egress IP
Budget observed	roughly 5–6 signups per window per IP
Effect	blocks the whole card matrix, because /payment needs a session

/payment redirects straight back to /signup without a session, so there is no way to reach the card form without signing up first. One account cannot cover the matrix either, because a successful card creates a subscription and the account then answers you already have an active subscription.

What would unblock it

1. Raise or disable the throttle on QA for the test window, or allowlist our egress IPs; or
2. Confirm the signup limit so we can pace a full run across it (a paced run over a couple of hours is entirely doable); or
3. Provide a QA route that reaches the payment form without a fresh signup.

Note: the 429 body leaks server file paths and line numbers via a debug block. Harmless on QA, worth confirming it is disabled anywhere public.

Confirmed working

Case	Result	Evidence
monthly × 1 seat, Visa (full trial)	PASS	429, 429, 201 signup then 200 from /api/subscribe, browser sent to qa.net-atelier.com/login. The complete happy path works end to end.
signup → payment	PASS	POST /api/auth/signup returned 201, browser sent to /payment?billing=monthly&seats=1
decline — generic	PASS	inline "your card was declined.", stayed on payment
decline — insufficient funds	PASS	inline "your card has insufficient funds."
decline — expired card	PASS	inline "your card has expired."
decline — incorrect CVC	PASS	inline "your card's security code is incorrect."
invalid expiry year (12/95)	PASS	inline "your card's expiration year is invalid."
invalid CVC (2 digits)	PASS	inline "your security code is incomplete."
empty email + password	PASS	blocked in the browser, no API request made
short password (<8 chars)	PASS	blocked in the browser, no API request made
3DS card 4000 0000 0000 3220	MANUAL	requires_action: true returned; the challenge itself needs completing by hand

Declines are handled well: every decline card produced a clear inline message and kept the user on the payment page. That is the behaviour you want.

Defects and flags

1. Malformed email is not validated in the browser — it costs a request

Submitting not-an-email or someone.example.com still POSTs to /api/auth/signup, and the rejection ("Please enter correct email.") comes back from the server. The form is marked novalidate, so there is no native check.

Why it matters: every malformed email consumes the same rate-limit budget as a real signup. Given the throttle above, typos actively use up the signup allowance. Fix: validate the format client-side before the request.

2. The Radar trial-abuse card was accepted

Test card 4001 8588 2184 3804 (free-trial-abuse) was accepted — /api/subscribe returned 200 and the browser reached qa.net-atelier.com/login. The expected Radar block did not happen.

This may simply mean the Radar rule is not enabled on QA rather than a code fault, so it needs a human to confirm which Radar rules are active. It is flagged, not asserted as broken.

Cannot be tested through the UI

Invalid expiry month cannot be driven from the browser. Stripe's expiry element silently rewrites the input: typing 13/34 results in a screenshot showing 01 / 33, which is a valid future date, so the card legitimately succeeds.

This one is worth calling out because it nearly became a false defect report: an earlier version of the runner scored it as "invalid card data was accepted", which was our harness being fooled, not the product. The runner now reads the field back and marks the case untested when Stripe has normalised the value.

Still to be tested

Full detail, per case, is on the published gallery under "Still to be tested". In summary:

• 8 cases need their own signup (annual × 1 seat, monthly/annual × 3/10 seats, Amex, Discover, seats clamping, off-session mandate). Blocked by the signup throttle. The one subscription case we did get through — monthly × 1 seat — passed.
• 9 cases need tooling a browser does not have: Stripe test clocks (advance 14 days), dunning and retry exhaustion, grace period expiry, customer-portal recovery, seat-change proration, upgrade paths, the cancel-during-trial endpoint, and webhook logs. These need test-mode Dashboard or CLI access.

How to re-run this

# from either host (fresh IP = fresh rate-limit bucket)
cd /opt/stripe-qa
python3 stripe_qa_runner.py --matrix matrix.json --out run --group shared
python3 stripe_qa_runner.py --matrix matrix.json --out run --group own

--group shared needs exactly one signup and covers every case that leaves the account untouched (declines, invalid data). --group own covers the cases that complete a subscription — one signup each. That split is what makes the run survivable under the throttle: 18 signups becomes 10, and the useful half needs one.