AI-Native Testing Strategy, in Practice
AI test generation is fast. The tests it produces can be almost useless. Here is how to use it well across unit, integration, and Playwright layers.
Testing is one of the areas where AI assistance produces the largest apparent productivity gain and the largest hidden risk.
The apparent gain is obvious. A developer who used to write tests by hand can now generate a full suite in minutes. Unit tests for a service class, integration tests for an API endpoint, Playwright specs for a user flow. What used to be an hour of work becomes a five minute prompt.
The hidden risk is that generated tests can look thorough and be almost useless. A test suite with high coverage numbers, green CI, and no actual assertion strength is worse than no tests at all, because it produces false confidence. Teams stop looking at the code carefully because the tests appear to be doing the job.
The general principle first
AI is good at scaffolding tests. It is not good at deciding which tests actually need to exist.
The most common failure pattern in AI generated test suites is that they test what the code does, rather than what the code should do. An AI asked to write tests for a function will read the function and produce tests that assert its current behaviour. If the current behaviour is wrong, the tests will confirm it as correct. If the function is missing an important case, the tests will not surface that gap.
This is why test generation should almost never happen before the developer has thought clearly about what the code needs to do. The direction of work matters. Specify the behaviour first, then use AI to accelerate the scaffolding, not the other way around.
Unit tests
Unit tests are where AI acceleration works best, and also where the risk of shallow coverage is highest.
Where AI helps
Setting up test scaffolding, mocking dependencies, generating multiple variations of similar test cases, producing boilerplate for testing frameworks with verbose syntax. The undifferentiated work of unit testing responds well to generation, particularly for classes and functions with clear inputs and outputs.
Where AI struggles
Identifying edge cases that are not obvious from the code, understanding which invariants matter more than others, and distinguishing between behaviour that must be preserved (regression protection) and behaviour that is incidental (over specification). A generated unit test suite tends to lock in the current implementation as if every detail of it were intentional, which makes future refactoring harder than it needs to be.
Practical approach: the developer defines what the unit is supposed to do, ideally before the implementation is written. They identify the specific behaviours worth asserting, including the edge cases they know from experience are likely to break. They then use AI to generate the mechanical parts of the test file: the setup, the mocks, the parameterised cases. They review the generated tests against their list of intended behaviours, remove tests that assert incidental implementation details, and add the edge cases AI missed.
A useful heuristic: if the AI generated a test and you cannot immediately say why that test is worth having, delete it. Coverage numbers are not the goal. Meaningful assertions are.
Integration tests
Integration tests are where AI assistance is genuinely useful, but the developer's involvement in shaping the tests matters much more than at the unit level.
Where AI helps
Setting up the shared fixtures, database seeding, HTTP client scaffolding, and the plumbing that makes integration tests possible. Also useful for generating the happy path assertions once the developer has specified what a successful interaction looks like.
Where AI struggles
Identifying the failure modes that matter. Integration tests need to cover what happens when things go wrong at the boundaries: the third party API returns a partial response, the database is temporarily unavailable, the message queue delivers a duplicate. These failure modes are specific to the system and often specific to production incidents the team has already had. An AI has no way to know which of the many possible failures are the ones actually worth testing.
Practical approach: the developer identifies the specific interactions that need integration coverage and, more importantly, the specific failure modes that are worth reproducing in tests. They are usually informed by the team's own history: outages, near misses, edge cases in upstream services that were expensive to discover the first time. AI is used to accelerate the writing of these tests once the list is defined.
A common mistake worth naming
Generating integration tests that cover the happy path only. The happy path is usually the least valuable path to test at the integration level, because it is the path most likely to be caught by unit tests and by the developer running the code locally. Integration tests exist primarily to catch failure modes that only manifest when real systems interact under realistic conditions. If the integration suite is mostly happy path, it is doing the wrong job well.
Playwright frontend testing
Playwright and similar frontend testing frameworks are the layer where AI generated tests are most likely to be superficially impressive and practically useless.
Where AI helps
Setting up the test project, generating page object models, writing the boilerplate of a test file, and producing tests that follow a straightforward user flow through the UI. Playwright's API is well documented and lends itself well to generation.
Where AI struggles
Stability, selector choice, and knowing which flows are actually worth automating. AI will happily generate tests with brittle selectors that break every time a class name changes, that assert on visual detail that is not what the user cares about, and that cover flows that would rarely fail in practice. Frontend tests are expensive to maintain, and a large suite of brittle tests is a net drain on the team's time, not a benefit.
Practical approach: the developer identifies the small number of critical user flows that actually need automated coverage. Log in and out. Complete a purchase. Submit the primary form. Whatever the small number of flows are, where a regression would be visible to real users and expensive to fix in production, those are the flows worth automating. Everything else should be caught at lower layers or by manual testing before release.
For each of those flows, the developer establishes the selector strategy first: prefer role based selectors, then data test attributes explicitly added for testing, then user visible text. AI generated Playwright tests often default to CSS selectors that are brittle by design. Fixing this after the fact is expensive. Setting the strategy first, and using AI within that constraint, produces tests that survive real product changes.
A specific rule worth adopting: if a Playwright test needs to be updated because the UI changed but the underlying feature did not, the test is written against the wrong signal. The test should assert on user visible behaviour, not on the specifics of how that behaviour is currently implemented. AI does not know this distinction unless you tell it.
The pattern across all three layers
The consistent pattern across unit, integration, and Playwright testing is this: AI accelerates the mechanical parts of test writing, and the developer's judgement on what to test remains the thing that actually matters.
Teams that lean into AI test generation without maintaining that judgement end up with large, brittle, superficially reassuring test suites that miss the failures that matter and break constantly on changes that should not affect them.
Teams that use AI as an accelerator for a thoughtfully specified test strategy end up with smaller, more meaningful, more stable suites that catch the failures worth catching and stay useful as the code evolves.
Practical checklist for teams adopting AI test generation
Before generating any tests
Specify what the code should do, separately from what it currently does. If those are not the same, fix the code before generating tests around the current behaviour.
For unit tests
Identify the specific behaviours worth asserting, generate tests for those, delete generated tests that assert incidental implementation details.
For integration tests
List the failure modes that matter, especially the ones from your team's own history, and prioritise those over happy path coverage.
For Playwright and frontend tests
Identify the small number of critical user flows worth automating, set the selector strategy before generating anything, and be willing to keep the suite small rather than large.
Across all layers
Coverage numbers are not the goal. Meaningful assertions are. A test that does not tell you anything when it fails, or does not fail when the code is wrong, is worse than no test at all.
The difference is not tool skill. It is testing skill. AI assistance amplifies whatever discipline the developer brought to the work in the first place. A developer who thinks carefully about what to test produces better AI accelerated test suites than a developer who does not, in exactly the same way that they would have produced better hand written test suites.
Read next
How to Review AI-Assisted Code Without Becoming the Bottleneck
AI has made developers more productive. That productivity is landing in your review queue. Most teams are not ready for it.
GuideHow AI-Native Developers Plan for Agentic Execution
A practical guide to writing specs for agentic work. When an agent is doing the work, gaps in the spec get filled silently.
Field ReportThe Developer Who Stopped AI Code Review
Not because it was not useful, but too agreeable. A field report on automation bias and staying engaged in review.
Want developers who work this way?
Every contractor we place uses AI tooling as a standard part of how they deliver. Tell us what your team needs.
Get in touch