Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve BDD with new testing command #420

Merged
merged 1 commit into from
Jul 8, 2024

Conversation

Art4
Copy link
Collaborator

@Art4 Art4 commented Jul 8, 2024

Using Behat for BDD and BDT is really impressive and useful. For new features I create a temporary tag @wip and run docker compose exec php composer behat -- --tags=wip. To test the feature on a specific redmine version I run docker compose exec php composer behat -- --tags=issue,group --suite=redmine=50103. If the feature is ready I run all behat tests on all redmine versions using docker compose exec php composer behat.

However the tests are becoming bigger and that leads to 1.) higher usage of time and memory to execute all tests at once. And 2.) it is not possible to change the format of the output using CLI (because the --output=progress is already declared in composer.json).

This PR solves these problems by creating a new bdt composer command for tests grouped by Redmine versions (aka behat suites). The output there is always formatted as progress. If one test fails it is now easier to find the Redmine version, where the test has failed thanks to the separation by behat suite.

Test-run: docker compose exec php composer bdt
$ docker compose exec php composer bdt
> behat --config tests/Behat/behat.yml '--format=progress' '--suite=redmine_50103'
...................................................................... 70
...................................................................... 140
...................................................................... 210
...................................................................... 280
...................................................................... 350
...................................................................... 420
...................................................................... 490
...................................................................... 560
...................................................................... 630
...................................................................... 700
...................................................................... 770
...................................................................... 840
..............................................

89 scenarios (89 passed)
886 steps (886 passed)
1m56.30s (15.69Mb)
> behat --config tests/Behat/behat.yml '--format=progress' '--suite=redmine_50009'
...................................................................... 70
...................................................................... 140
...................................................................... 210
...................................................................... 280
...................................................................... 350
...................................................................... 420
...................................................................... 490
...................................................................... 560
...................................................................... 630
...................................................................... 700
...................................................................... 770
...................................................................... 840
..............................................

89 scenarios (89 passed)
886 steps (886 passed)
2m9.03s (15.66Mb)
> behat --config tests/Behat/behat.yml '--format=progress' '--suite=redmine_40210'
...................................................................... 70
...................................................................... 140
...................................................................... 210
...................................................................... 280
...................................................................... 350
...................................................................... 420
...................................................................... 490
...................................................................... 560
...................................................................... 630
...................................................................... 700
...................................................................... 770
...................................................................... 840
..


84 scenarios (84 passed)
842 steps (842 passed)
1m59.39s (15.57Mb)

The behat command will be changed to have no output format (defaults to --format=pretty). So if I run docker compose exec php composer behat -- --tags=wip I will get the pretty format that helps me with debugging. If I want to have the progress output I can declare it with docker compose exec php composer behat -- --tags=wip --format=progress.

This PR also improves the test docs explaining the different commands. I've tested theses new commands while working on #419.

@Art4 Art4 added this to the v2.7.0 milestone Jul 8, 2024
@Art4 Art4 requested a review from kbsali July 8, 2024 10:42
@Art4 Art4 self-assigned this Jul 8, 2024
Copy link

codecov bot commented Jul 8, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.54%. Comparing base (dfa6b49) to head (e502824).

Additional details and impacted files
@@            Coverage Diff            @@
##               v2.x     #420   +/-   ##
=========================================
  Coverage     98.54%   98.54%           
  Complexity      639      639           
=========================================
  Files            29       29           
  Lines          1925     1925           
=========================================
  Hits           1897     1897           
  Misses           28       28           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Art4 Art4 merged commit e31143f into kbsali:v2.x Jul 8, 2024
12 checks passed
@Art4 Art4 deleted the improve-commands-for-bdt branch July 8, 2024 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants