Let AI Test Its Own Work: A UAT Loop With Claude Code and Playwright

Yet another tip from how I actually work with Claude Code: let AI test your own plan.

Not just write it. Test it.

Start With a UAT File

If you write a spec, write a UAT file too — User Acceptance Testing, that’s what it stands for. Then you just tell it: test this using Playwright, authenticate using the browser, and so on and so forth.

That’s the whole instruction.

Give It the Keys (Locally Only)

Here’s the part that makes people nervous: to actually test something, AI needs to log in as different users. So you put the credentials in your .env file.

And by the way — this is only stored locally, so don’t worry about this one.

You give it the credentials to test all the examples. In my case, I gave it admin credentials so it can test using admin. I gave it user credentials. And then I gave it professional credentials, because in my product these are different types of personas.

Ask for a Verdict, Not Just a Report

Then I ask AI to test and comment on my file — the UAT file, let’s say — with an emoji. A green tick to say yes, a red one to say no, that kind of stuff.

And then you can iterate.

I’ve been following this practice for a while now: I ask AI to test the majority of the stuff, and then I do the rest myself.

The Next Level: Fix It, Then Retest

Here’s yet another tip, one level up. Once AI has run the tests, you can do something more.

You can say: “Hey, with your discovery, please fix it and then retest.”

That’s useful because, almost autonomously, AI will find, will test, will solve, will retest, and will get back to you.

I mean, this feels like magic. But it is — and it’s super useful.

I Turned This Into Two Skills

I got tired of walking through this by hand every time, so I packaged the whole loop into two Claude Code skills. They work as a pair. One writes the UAT file to a strict format. The other reads that file, drives the browser, and fills in the results. Define produces the file. Execute consumes it. Neither one does the other’s job, and that’s kind of the point — I can drop both into any project and they just work.

uat-define — writes the file, never runs anything.

---
name: uat-define
description: Write a UAT (User Acceptance Test) file to a strict, reusable format. Does NOT run tests — writing only.
---

## Where it lives
One file per feature: `docs/backlog/uat/uat-<feature>.md` (kebab-case). Adjust the folder
to your project if different, just be consistent.

## Header (in order)
1. `# UAT — <Feature>`
2. Purpose — 2-4 lines, what this file verifies and why.
3. Prerequisites (blockquote, numbered) — external dependencies and what breaks if missing.
   This is how the executor later decides between ⚪ and 🔴.
4. Test accounts — reference **.env variable names only** (e.g. `ADMIN_USERNAME` /
   `ADMIN_PASSWORD`). Never write plaintext credentials.
5. Status legend (always present, always identical):
   - (no icon) — still to test
   - 🟢 tested & passed manually
   - 🤖 tested & passed by an automated test
   - 🔴 tested & failed
   - 🟡 tested but something's off (partial, needs review)
   - ⚪ test attempted but couldn't reach the step (missing dependency/data, blocked upstream)
6. Optional: a short summary of business rules to verify.

## Structure
- Sections by LETTER: A, B, C… Typically one section per user type or scenario
  (e.g. A = anonymous user, B = logged-in user, C = admin). Name the test account for
  the section right in its heading or intro.
- Each section has NUMBERED sub-sections: `### A1 · <title>`, `### A2 · <title>`.
- Every test point gets a unique, stable hierarchical code: `A1.1`, `A1.2`, `A3.4`, `B2.1`.

## Hard rules
- One check per bullet. Never bundle two assertions into one point.
- Codes are unique across the file and **never renumbered**. Adding a point later? Use
  the next free number, or a decimal (`A1.6`) — never shift existing codes.
- Keep sections consistent: if one has sub-sections, all of them do. No loose points
  floating outside a sub-section.
- If a point needs a state no test account has (e.g. "a user who already made a
  purchase"), say so in the point text itself, so execute knows to ask instead of guessing.
- Every point is born with **no status icon**. That's the default: still to test. Define
  never sets status, that's execute's job.
- Write points in the imperative, concrete, binary: "Open the **Settings** tab.",
  "Click **Save**.", "Verify that the confirmation banner appears." Bold key UI terms,
  use `code` for routes/identifiers.

## Definition-done checklist
- [ ] File in the right folder with the right name
- [ ] Complete header: purpose, prerequisites (if any), test accounts, status legend
- [ ] Lettered sections + numbered sub-sections, no loose points
- [ ] Every point has a unique code and no icon
- [ ] Codes are unique and stable across the file
- [ ] A test account is named per section, no plaintext credentials anywhere

uat-execute — runs the file, never writes one from scratch.

---
name: uat-execute
description: Run an existing UAT file — drive the browser, verify each point, write status back into the file. Requires a UAT file as input; never invents or restructures one.
---

## Preflight (always, before anything)
1. Read the whole UAT file.
2. Parse every point (code, section, sub-section), the status legend, prerequisites,
   and the test account named per section.
3. If the file doesn't match the uat-define format, stop and say so. Suggest fixing it
   with uat-define first. Don't try to test a malformed file.
4. If a prerequisite is clearly unmet, the dependent points get ⚪ with a note, not 🔴.
   That's not a feature failure, it's a test that can't run yet.

## No file, no run
Execute never starts without a path to an existing UAT file, and it never invents or
restructures one, that's uat-define's job. If you weren't given a path, ask which file,
or confirm the one already open/cited before touching it.

## Server URL: detect and confirm, never assume
This is the rule that bites people. Test the server the user is actually running, not
a guessed default.
1. If the user gave an explicit URL/port, use it.
2. Otherwise, detect listening servers (check running node/vite/next processes) and
   cross-check against project config (`.env` port variables, dev server config,
   package.json scripts).
3. Always confirm the exact URL with the user before testing, especially if you found
   more than one candidate, or none. Never default to a random port like 8080.
4. Keep that confirmed URL for the whole session and say which one you're using.

## Browser
Drive it via Chrome DevTools MCP: navigate, take a snapshot to read the DOM/accessibility
tree (preferred way to assert state), take a screenshot for visual proof or doubtful
cases, click, fill, wait. Reuse the same open browser session, don't spin up a new one
per point. Switch accounts only when the section changes.

## Login, per section
- Accounts live in the project's `.env`. Read the values at runtime, never print them
  in files, commits, chat, or logs.
- Log in with the account named for that section, run every point in the section, then
  move to the next.
- Use the "cleanest" available account (no prior purchases/entitlements) for anything
  paywall- or purchase-related.
- Logged-out cases run in an unauthenticated context.

## Classify before you touch a point
- Testable on your own → run it, set the status.
- Needs a real manual action (real payment, OTP, a DB flag someone has to flip) →
  **stop and ask**. Don't simulate what you can't actually do.
- Needs an account state the test accounts don't cover → ask for the right setup, or
  mark ⚪. Don't pass off a case you didn't properly set up as green.
- Blocked by a missing prerequisite → ⚪ with a note, don't attempt it.

Batch it: get through everything you can do autonomously first, then come back with
ONE list of questions, not one interruption per point.

## Writing status back
Edit the file in place. Icon goes right after the code:
`- **A1.2** 🟢 Open the tab…`

- 🟢 verified by hand in the browser, passes.
- 🤖 covered and passed by a real automated test, only if one actually exists, cite it
  in the note.
- 🔴 executed, fails.
- 🟡 works, but something's off, note what.
- ⚪ couldn't reach it, note what's missing.
- No icon = not touched yet.

Add a short note for every 🔴/🟡/⚪ (what you saw, or what's missing). Never touch the
point's original text or its numbering, only the icon and the note. Save the file
incrementally as you go, not just at the end.

## Finish
One summary: count per status, list every 🔴 and 🟡 with its note, and say what's needed
to unblock the remaining ⚪. Never auto-commit, that's the user's call.