What to automate first: how to choose your first 20 regression tests

Most teams don't fail at test automation because they picked the wrong tool. They fail because they try to automate everything at once, or they start with whatever is easiest. Months later, they have a long suite that nobody trusts and that still misses the bugs customers find.

A better start is small and deliberate: 20 well-chosen regression tests. The number is a rule of thumb, not a magic figure. It's small enough to finish and maintain, but big enough to protect the parts of your product that matter most.

Here's how we would choose them. 👇

What should you automate first?

Automate the checks that are important, repeated often and stable. In practice, that usually means:

• Revenue journeys: sign-up, checkout, payment and subscription changes.
• Access journeys: login, logout and password reset.
• Core product actions: the main thing customers came to do.
• Painful past bugs: anything that has broken more than once.

If a failure would wake someone up at night or lose a customer, it belongs near the top of your list.

What should you leave out for now? 🚫

Automation isn't the right answer for every check. Leave these out of your first batch:

• Features that are still changing every week.
• Rare admin tasks with low business risk.
• Visual details that need human judgment.
• One-off checks you'll never repeat.
• Exploratory testing, where the value comes from a person asking "what if?"

Score your tests in 10 minutes 🧮

List your candidate tests, then score each one from 1 to 3:

• Impact: how much damage would a failure cause?
• Usage: how often do customers use this?
• Stability: how settled is this feature?

Add the scores together. Start with anything scoring 7 or more. Here's an example for an online store:

Test	Impact	Usage	Stability	Score	Decision
Checkout with a card	3	3	3	9	✅ Automate now
Login	3	3	3	9	✅ Automate now
Password reset	3	2	3	8	✅ Automate now
Search filters	2	3	2	7	✅ Automate now
New homepage banner	1	3	1	5	⏳ Wait
Admin CSV export	2	1	2	5	⏳ Wait

9 tips for a regression suite that lasts 💡

1. Let real data choose for you

Don't rely on guesses alone. Check your analytics, bug history and support tickets. They show which journeys customers really use and where problems keep coming back.

2. Build a small smoke suite first 🔥

Pick about five of your most critical tests and run them on every pull request. They should finish quickly and answer one question: is the product basically working?

3. Test at the lowest level that works

Not every rule needs a browser. Check pricing, validation and permissions through the API, where tests are faster and more stable. Our guide to API testing with Jest and Supertest shows how. Save browser tests for full customer journeys.

4. Give each test one clear purpose 🎯

A test that covers sign-up, checkout and account settings fails for many different reasons. Keep each test focused, so a failure tells you exactly where to look.

5. Let every test own its data

Shared data is a common cause of flaky tests. Each test should create what it needs and clean up afterwards. This also makes running tests in parallel much safer.

6. Use locators that survive redesigns

Find elements the way a user would, by role, label or visible text. Long CSS paths break as soon as the layout changes, even when the feature still works.

7. Run tests in CI/CD from day one ⚙️

Tests that only run on one person's laptop get forgotten. Connect the suite to your pipeline early, and let a failed test block the release.

8. Fix or remove flaky tests quickly 🧹

Once people get used to rerunning failed tests, they stop trusting the suite. Fix a flaky test quickly, or take it out until you can. Running tests in random order, with options like --shuffle in Playwright 1.64, can reveal hidden dependencies between them.

9. Review your suite every month 🔁

Your product changes, so your suite should too. Add tests for new risks and for bugs that got through, and delete tests that no longer earn their place.

A simple two-week plan 🗓️

1. Days 1 to 2: List candidate tests and score them.
2. Days 3 to 5: Set up the framework and build the smoke suite.
3. Days 6 to 8: Connect it to CI/CD and make the first runs stable.
4. Days 9 to 10: Finish the remaining tests and agree who owns the suite.

Your timeline will depend on your product and team, but the order holds: prove that a small suite is reliable before you make it bigger.

Common mistakes to avoid ⚠️

• Automating the easiest tests instead of the most important ones.
• Measuring success by the number of tests.
• Letting one person hold all the knowledge.
• Ignoring failed runs because "that test always fails."
• Treating automation as a replacement for human testing.

Frequently asked questions

What should be automated first in testing?

Start with stable, high-impact journeys that customers use often, such as login, sign-up and checkout, along with areas that have broken before.

How many regression tests should a team start with?

Around 20 is a practical starting point for many web products. The right number depends on your product's size and risk, so focus on covering your most important journeys rather than hitting a target.

What tests should not be automated?

Avoid automating fast-changing features, one-off checks, low-risk rare tasks, visual details that need human judgment, and exploratory testing.

What is the difference between smoke and regression testing?

A smoke test quickly checks that the most critical functions work. A regression suite goes further and checks that recent changes haven't broken existing behavior.

How often should regression tests run?

Run a small smoke suite on every pull request and the full regression suite at least nightly or before each release, depending on how often you ship.

Not sure where your first 20 tests should start? See how our test automation services work, or start with a free test run on your web application. 🚀