Allure Reports

Allure Report turns automated test results into reports for exploring scenarios, steps, parameters, and attachments. Its practical value is shortening the path from a red status to an understood cause. Attractive charts alone cannot make weak checks reliable.

Distinguish the runner, adapter, and report generator. Pytest or JUnit executes tests, an adapter records structured results, and Allure builds a view from those files. A missing report and a failed test are separate events requiring separate diagnosis.

What readers need to see

A useful report identifies the tested build, environment, expected result, and actual outcome. For an order, an operation ID and server response matter more than dozens of “call helper” steps.

Names should describe behavior. “A customer cannot read another customer's order” explains a risk; “test_17” requires opening the code. Parameters distinguish variants, but random technical UUIDs usually belong in diagnostics unless they are central to the case.

A Pytest example

This training project requires a supported Python version, pytest, allure-pytest, and a separately installed Allure Report 3 CLI with a suitable Node version, following its instructions. Allure 2 and Allure 3 have different installation procedures and commands; this example uses Allure 3. Installing the pip adapter does not automatically install the full CLI. Pin adapter and CLI versions after checking their requirements.

Save the code as test_report.py. It independently demonstrates report structure using a synthetic object, rather than testing a real store. In a product, replace order retrieval with access to your system and isolated test data.

python

import json
import allure

@allure.title('Draft order contains the prepared total')
def test_report_example():
    with allure.step('Prepare a synthetic order'):
        order = {'id': 'demo-42', 'status': 'draft', 'total': 9000}
    with allure.step('Attach safe diagnostic data'):
        allure.attach(json.dumps(order), name='order',
                      attachment_type=allure.attachment_type.JSON)
    with allure.step('Check the expected values'):
        assert order['status'] == 'draft'
        assert order['total'] == 9000

After installing the adapter, run the commands below. Use a new empty results directory for every independent run; run-001 identifies this example. The generate command writes to an explicit report directory; open starts a local viewer and may remain active until stopped with Ctrl+C.

bash

python -m pip install pytest allure-pytest
python -m pytest test_report.py --alluredir=allure-results-run-001
allure generate allure-results-run-001 -o allure-report-run-001
allure open allure-report-run-001

Expect one passing test with three meaningful steps and a JSON attachment. For a diagnostic exercise, change the expected amount to 8,000: the comparison should fail and the report should display the actual order. Restore the agreed expectation afterward.

Steps should explain actions

Good steps expose meaningful boundaries: prepare a cart, apply a coupon, and read the result. One step per technical line creates noise; one step for the whole test hides the failure location. Choose detail according to its diagnostic value.

Expected values must come from requirements. A step name does not establish assertion correctness. Comparing results with the same calculation used by the application can let a shared error produce an impressive but useless green report.

Which attachments to retain

For UI tests, retain a screenshot and trace or another appropriate tool artifact. For APIs, retain sanitized requests, responses, status, and correlation id. For mobile scenarios, retain application version, device, and logs around the failure. Capture context at the first significant violation.

Keep passwords, tokens, cookies, and unnecessary personal data out of reports. Mask before writing files: interface access restrictions do not remove source files. Define artifact size and retention policies too; unlimited video storage quickly becomes expensive.

Statuses need interpretation

Status depends on the runner and adapter. Assertion failures and setup errors may appear differently; learn your integration's rules. First determine what failed: a product outcome, fixture, infrastructure operation, or report generation.

Skipped tests cannot count as passed. A successful retry also does not erase the initial failure. Release decisions need visibility into final outcomes, retries, and exclusions, or a reassuring report can hide unreliable testing.

History and test identity

Comparisons across runs require stable test identity and correct history transfer according to the chosen Allure version's instructions. Random names or continuously changing parameter structures can split one check into many unrelated records.

History reveals instability and duration growth but does not automatically explain causes. Timing changes may come from the test, environment, or data. Compare trends alongside application and environment versions while preserving comparable run meaning.

CI integration

Collect results and publish artifacts after test failures whenever the agent can still do so. Preserve the runner's exit code: successful HTML generation must not turn a failed test job green.

Keep results from different builds in separate directories. Leftover files can add old cases and distort totals. Record the commit, environment, and start time, and place the report link beside the corresponding job result.

Investigating a failure in practice

Suppose an API returns an order with draft status instead of paid. Open the precise assertion and sanitized response first. Compare operation and correlation IDs with processing logs. If payment is still running, check the intended wait for asynchronous completion; if it failed, investigate that failure.

A report is useful when another team member can start this investigation without contacting the test author. If an essential ID is missing, add it to an attachment before adding another overall pass-rate chart.

Common mistakes and lessons

Common problems include stale result files, hidden exit codes, secrets in attachments, and excessive step detail. Another mistake is treating Allure Report as a test management system and expecting one HTML report to manage requirements completely.

Allure helps when tests have clear expectations and collect relevant evidence. Organize reports around investigation and team decisions: what failed, how much confidence the result deserves, and what action comes next.