Hand this to your AI agent. It does the boring part.

Let Claude Code fix failing Playwright tests overnight (draft PR)

Shared from yuden (Skillnote), who ran a scheduled Playwright nightly that hands failures to Claude Code, then opens a draft pull request for a person to review.

1. 1Protect main and decide draft-only (human checkpoint)you check›

   Require pull request reviews on the default branch. yuden’s flow creates Draft PRs and does not auto-merge. Keep it that way. Sources: GitHub protected branches; yuden “運用フロー” (Draft PR + QA review).
2. 2Install Claude Code GitHub Actions auth (docs-backed)›

   Follow Anthropic’s Claude Code GitHub Actions setup: Claude GitHub App (or custom app), repository secret (ANTHROPIC_API_KEY or CLAUDE_CODE_OAUTH_TOKEN), workflow under .github/workflows/. Prefer pinning anthropics/claude-code-action to a commit SHA. Do not copy kickflow’s claude_args: '--dangerously-skip-permissions'. That turns off interactive permission checks. yuden’s run used an allowlist instead.
3. 3Trigger after a failed nightly (pattern from the run)›

   yuden: scheduled Playwright nightly (they cite 85 spec files) saves a Playwright JSON report as an artifact; on failure, _analyze-and-fix.yml (reusable workflow_call, also workflow_dispatch) starts. Official example ci-failure-auto-fix.yml uses workflow_run on CI failure — read its SECURITY NOTE (PR branch code runs with write permissions; same-repo only; trusted writers only).
4. 4Narrow tools the way the run did›

   yuden’s analyze step used Claude Code CLI roughly as in the prompt below. They note the sample is still broader than production should be — tighten further. Limit edit paths in the prompt (they restricted to specs/, pages/, utils/). Official docs: pass --allowedTools via claude_args on claude-code-action.
5. 5Prompt: classify, minimal fix, quality gate, draft PR›

   yuden’s five-step prompt shape (paraphrased): collect playwright-report.json + failed logs; classify fixable-in-test vs server bug using a checked-in instructions file; branch + minimal edit; run npm run code-quality until it passes; commit, push, open Draft PR. On agent failure, open a GitHub Issue with the failed run link (their fallback). What their run showed as a real fix: a settings table with rowspan put “delete” on a different <tr> than “register”; the old locator used ancestor::tr from the category cell and missed delete. Claude rewrote the XPath to start from the operation cell and allow the category on the same row or the previous sibling row. Merge decision stayed with humans.
6. 6Review every PR in the morning (human checkpoint)you check›

   Read the diff. Reject timeout-only “fixes,” edits outside the allowed paths, or skipped assertions. yuden flags that Claude sometimes lengthens timeouts — their instructions forbid changing default poll values; review still catches misses. - Draft PR with analysis (or Issue if the agent aborts). - Human merge only after review. - First weeks: compare proposed locators against the app under test yourself.
7. 7Who it helps and what it accomplishes›

   Morning triage of flaky or selector-broken E2E eats time. yuden’s team runs Playwright on a schedule; when the run fails, a follow-on workflow starts Claude Code, which reads the report and logs, applies a minimal fix under path limits, and opens a draft pull request. A person reviews and merges. They report two auto-heal activations that were mergeable after review, and a concrete fix for a rowspan table locator that had been matching the wrong row.
8. 8Failure cases›

   No PR: Permissions, secret, allowed_bots / actor checks in the Action docs. Timeout-only diffs: Reject; tighten instructions. Edits outside allowlist: Path rules in prompt + review. Fork PRs: Official auto-fix example warns about executing PR branch code — do not enable for untrusted contributors