Playwright · Reliable end-to-end tests
How do you diagnose and fix a flaky test?
Short answer
Treat flakiness as a bug. Reproduce it with repeats and traces, then remove the race: use web-first assertions, isolate data, control the network and clock, and avoid fixed sleeps.
A flaky test has non-deterministic inputs or timing. The goal is not to make it pass more often; it is to make the outcome deterministic. That means finding the race or shared state and removing it, not adding a longer wait.
Common causes
- A fixed sleep that is sometimes too short for the environment
- Asserting on DOM before a web-first assertion has settled
- Tests sharing an account, a database row or a file
- Dependence on test order or on a previous test's side effects
- Time, locale, timezone or random data that differs between runs
- Animations, third-party scripts or unstable network responses
npx playwright test tests/checkout.spec.ts --repeat-each=20
npx playwright test tests/checkout.spec.ts --repeat-each=20 --workers=1
npx playwright show-trace test-results/checkout/trace.zipawait expect(page.getByRole("button", { name: "Publish" })).toBeEnabled();
await page.getByRole("button", { name: "Publish" }).click();
await expect(page.getByRole("status")).toHaveText("Published");Fix the most likely cause first, then repeat the test many times with one worker to rule out parallel collisions, and with several workers to expose shared-state issues. If the failure only appears in CI, compare browser version, timezone, locale, server latency and data seeding.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1Why is a retry not a fix for flakiness?
- 2How do you reproduce a flaky test locally?
- 3What role do time and locale play in flakiness?
- 4When is quarantine acceptable?
Say it in the interview
I treat flakiness as a determinism bug, not a timing annoyance. I reproduce it with repeat-each, inspect the trace, and look for fixed sleeps, shared data, order dependence or uncontrolled time and network. Then I replace the race with web-first assertions, isolated data and controlled responses. Retries and quarantine only buy time; they do not remove the cause.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live Playwright mock interview and get scored feedback on depth, clarity and confidence.