Summary

We’ve encountered an issue when running tests using the Allure Java adapter in environments where test execution uses some form of caching (e.g., Maven build cache with Github caching).

Problem

When a test is executed and results are generated, and a subsequent execution modifies the test name, title, or metadata, but reuses the previous cache, the old results are not fully cleaned from the output folder.

As a result, stale or duplicated test results may appear in the Allure report or be sent to Tesults or similar tools.

Steps to Reproduce

1. Run a test suite with a test named TestExampleA, generating Allure results.
2. Modify the test code: rename the test to TestExampleRenamed or change metadata (e.g., severity, description, etc.).
3. Re-run the test suite without clearing the results folder manually, using cached test output.
4. Observe that results from both TestExampleA and TestExampleRenamed appear in the output.

Expected Behavior

The allure-results or Tesults result folder should be cleaned automatically before writing new test results, to avoid outdated or duplicate entries.

💡 Suggestion: Add a clean option similar to --clean-alluredir in pytest

In the Allure Pytest integration, there's a very helpful --clean-alluredir option that clears the results directory before test execution starts (docs reference).

We suggest implementing a similar mechanism in the Java integration (e.g., allure-java-commons), to allow users to automatically clean the results folder (allure-results, Tesults folder, etc.) before writing new data.

What Allure Integration are you using?

allure-java-commons

What version of Allure Integration you are using?

2.26.0

What version of Allure Report you are using?

2.12.0

Code of Conduct

• I agree to follow this project's Code of Conduct