Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for selective test run with a test plan #125

Open
delatrie opened this issue Feb 12, 2024 · 0 comments
Open

Add support for selective test run with a test plan #125

delatrie opened this issue Feb 12, 2024 · 0 comments

Comments

@delatrie
Copy link

delatrie commented Feb 12, 2024

There is a general mechanism used by Allure adapters to run tests selectively. It uses a test plan - a .json file with the following schema:

{
    "type": "object",
    "properties": {
        "tests": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "id": { "type": "string" },
                    "selector": { "type": "string" }
                }
            }
        }
    }
}

A path to a test plan is provided as a value of the ALLURE_TESTPLAN_PATH environment variable.

If a valid test plan file is provided, only matched tests should be run. A test is matched if there is at least one entry in the tests array, such that:

  1. If the id property is present in the entry AND the test has an Allure ID (added via the allure_id or as_id label), the id property must be the same as the Allure ID of the test, OR
  2. If the selector property is present in the entry, it must be the same as the test's fullName.

Motivation

Test plan support allows selective runs from TMS (e.g., TestOps). Ideally, the mechanism should be supported by all Allure adapters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant