When to Automate

How to identify checks worth automating and those better handled manually or moved to another level.

Updated:

Contents

Automation makes sense when it answers an important question repeatedly, faster and more reliably than manual checking. A script that clicks buttons does not provide that value by itself. You need to understand the risk it controls, when it runs, and who investigates its result.

Make the decision for a specific check. A person is better suited to exploring a new document editor, while a program can recalculate a hundred pricing combinations after every change. These tasks complement each other: exploration discovers new risks, while automation protects understood behavior.

Start with risk and repeatability

A good candidate relates to meaningful harm: an incorrect order amount, access to another user's document, or inability to reset a password. It should run often enough to justify creation and maintenance. An infrequent operation may also qualify when a failure is critical and difficult to detect manually.

State the goal in plain language: "a customer does not receive the discount twice after resubmitting." Then define inputs and observable results. "Automate the cart" is too broad to estimate cost or identify which defect would remain undetected.

Can you determine the result reliably?

An automated test needs an oracle: a way to distinguish correct from incorrect behavior. This could be a formula, API contract, order state, or specific message. If the team is still debating correct behavior, clarify the rule first. A script cannot resolve conflicting requirements.

Not every qualitative judgment should become a Boolean answer. The usability of a complex workflow, persuasive text, and understandable navigation require context. An automated check can expose part of a problem, such as an inaccessible button, but cannot prove that the interface is easy for people to use.

Choose the narrowest sufficient level

A discount calculation is usually best tested near the business logic. HTTP authorization belongs at the service level. Browser tests are appropriate when the risk involves the interface or the connections across a user flow. The same boundary values do not necessarily need repeating at every level.

This choice affects diagnosis. When a formula fails in an isolated test, the cause is localized. When a long browser flow detects it, investigators first have to rule out the network, session, page loading, and test data. However, a narrow test does not replace confirming that the intended formula is actually connected to the product.

Assess control over the environment

Before implementation, determine whether you can create a separate user, prepare data, reset state, and obtain diagnostic logs. A shared account, manual order confirmation, and a constantly changing catalog make even a simple scenario expensive.

Often the best initial investment is a test data setup API, stable element identifiers, or an accessible event log. This improves several checks at once. Trying to work around missing capabilities with a long UI sequence usually transfers the problem into maintenance.

Practical example: applying a coupon twice

A training store has a rule: SAVE10 reduces an eligible order's cost by ten percent, once. A cart worth 10,000 currency units, a test user, and a separate environment without real payments are available. The team is concerned about resubmission after a delayed response.

At function level, test amounts of 10,000 and 0, ineligible products, and rounding according to the agreed rule. At API level, apply the same coupon to the same cart twice. Expect a total of 9,000 after both requests and one discount application record. Define the repeated response's specific status in the contract beforehand.

One browser scenario checks that the user sees the updated total and cannot obtain a double discount by clicking again. Use explicit state expectations and separate data. If the UI fails while the API passes, investigate rendering or request submission rather than rewriting the formula.

Run a limited pilot

Choose several candidates of different complexity and get them running regularly. Measure creation effort, average investigation time, false failures, and change costs. The pilot must include a real product change cycle: an initial green run says little about future maintenance.

Define success before starting: for example, the check runs on every change request, identifies a useful failure cause, and fits the feedback-time budget. Choose numbers from team needs rather than copying another project's targets.

What to leave to people for now

A new, unstable scenario can first be explored manually and clarified through examples. A one-time import may need only a dedicated reconciliation script rather than a permanent suite. Test external systems through agreed test environments and controlled responses, while retaining separate integration checks.

CAPTCHA or hardware confirmation is a reason to discuss test architecture. Use an authorized test mechanism in your own environment rather than trying to circumvent another service's protection.

Common selection mistakes

Automating all manual cases in sequence inherits their repetition and unclear expectations. Selecting only easy scenarios risks impressive test counts without controlling important losses. Starting with the most complex E2E test delays useful feedback.

Another mistake is treating execution as free. Data, infrastructure, library upgrades, false failures, and investigations take time. A check's value gradually declines if it has no owner or clear response to failure.

Revisiting the decision

Reassess a check after product changes. Remove obsolete risks, split overly broad tests, and reduce duplication. Deleting an irrelevant scenario is normal suite maintenance.

Strong automation starts with a clear risk, controllable conditions, and an observable result. Select checks the team will regularly use to make decisions, then verify that assumption against actual costs and useful findings.

Further reading