Skip to content

Commit

Permalink
Merge pull request #91 from jakubch1/main
Browse files Browse the repository at this point in the history
Add scenario 4 for testing platform and fix reports
  • Loading branch information
jakubch1 authored Jan 19, 2024
2 parents 7ccd598 + 05e5420 commit b853d98
Show file tree
Hide file tree
Showing 17 changed files with 359 additions and 200 deletions.
44 changes: 44 additions & 0 deletions .github/workflows/Algorithms_Scenario04.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# This workflow will build a .NET project
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-net

name: "Algorithms Scenario 04"

on:
push:
branches: [ "main" ]
paths: [ 'samples/Algorithms/tests/**', 'samples/Algorithms/src/**', '.github/workflows/Algorithms_Scenario04.yml' ]

jobs:
build:

runs-on: ubuntu-latest
defaults:
run:
working-directory: ./samples/Algorithms/tests/Algorithms.Core.Tests
steps:
- uses: actions/checkout@v3
- name: Setup .NET
uses: actions/setup-dotnet@v3
with:
dotnet-version: 8.0.x
- name: Restore dependencies
run: dotnet restore
- name: Build
run: dotnet build --no-restore
- name: Install dotnet-coverage
run: dotnet tool install -g dotnet-coverage
- name: Test
run: dotnet-coverage collect --output $GITHUB_WORKSPACE/report.cobertura.xml --output-format cobertura "dotnet run --no-build"
- name: ReportGenerator
uses: danielpalme/[email protected]
with:
reports: '${{ github.workspace }}/report.cobertura.xml'
targetdir: '${{ github.workspace }}/coveragereport'
reporttypes: 'MarkdownSummaryGithub'
- name: Upload coverage into summary
run: cat $GITHUB_WORKSPACE/coveragereport/SummaryGithub.md >> $GITHUB_STEP_SUMMARY
- name: Archive code coverage results
uses: actions/upload-artifact@v3
with:
name: code-coverage-report
path: ${{ github.workspace }}/report.cobertura.xml
3 changes: 2 additions & 1 deletion samples/Algorithms/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,5 @@ Solution contains two projects:

1. [***Scenario 01*** Code coverage for MSTest Runner project using dynamic instrumentation](scenarios/scenario01/README.md)
2. [***Scenario 02*** Code coverage for MSTest Runner project using static instrumentation](scenarios/scenario02/README.md)
3. [***Scenario 03*** Code coverage for MSTest Runner project using compile-time instrumentation](scenarios/scenario03/README.md)
3. [***Scenario 03*** Code coverage for MSTest Runner project using compile-time instrumentation](scenarios/scenario03/README.md)
4. [***Scenario 04*** Code coverage for MSTest Runner project using `dotnet-coverage` tool](scenarios/scenario04/README.md)
Binary file modified samples/Algorithms/scenarios/scenario01/azure-pipelines.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -1,74 +1,6 @@
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<coverage line-rate="1" branch-rate="1" complexity="15" version="1.9" timestamp="1704372866" lines-covered="34" lines-valid="34" branches-covered="12" branches-valid="12">
<coverage line-rate="1" branch-rate="1" complexity="10" version="1.9" timestamp="1705670662" lines-covered="17" lines-valid="17" branches-covered="10" branches-valid="10">
<packages>
<package line-rate="1" branch-rate="1" complexity="5" name="Algorithms.Core.Tests">
<classes>
<class line-rate="1" branch-rate="1" complexity="5" name="Algorithms.Core.Tests.MergerTests" filename="/home/runner/work/codecoverage/codecoverage/samples/Algorithms/tests/Algorithms.Core.Tests/MergerTests.cs">
<methods>
<method line-rate="1" branch-rate="1" complexity="1" name="Test1" signature="()">
<lines>
<line number="13" hits="1" branch="False" />
<line number="14" hits="1" branch="False" />
<line number="15" hits="1" branch="False" />
<line number="16" hits="1" branch="False" />
</lines>
</method>
<method line-rate="1" branch-rate="1" complexity="1" name="Test2" signature="()">
<lines>
<line number="20" hits="1" branch="False" />
<line number="21" hits="1" branch="False" />
<line number="22" hits="1" branch="False" />
<line number="23" hits="1" branch="False" />
</lines>
</method>
<method line-rate="1" branch-rate="1" complexity="1" name="Test3" signature="()">
<lines>
<line number="27" hits="1" branch="False" />
<line number="28" hits="1" branch="False" />
<line number="29" hits="1" branch="False" />
<line number="30" hits="1" branch="False" />
</lines>
</method>
<method line-rate="1" branch-rate="1" complexity="2" name="AssertArrays" signature="(int[], int[])">
<lines>
<line number="33" hits="1" branch="False" />
<line number="34" hits="1" branch="False" />
<line number="35" hits="1" branch="True" condition-coverage="100% (2/2)">
<conditions>
<condition number="0" type="jump" coverage="100%" />
</conditions>
</line>
<line number="36" hits="1" branch="False" />
<line number="37" hits="1" branch="False" />
</lines>
</method>
</methods>
<lines>
<line number="13" hits="1" branch="False" />
<line number="14" hits="1" branch="False" />
<line number="15" hits="1" branch="False" />
<line number="16" hits="1" branch="False" />
<line number="20" hits="1" branch="False" />
<line number="21" hits="1" branch="False" />
<line number="22" hits="1" branch="False" />
<line number="23" hits="1" branch="False" />
<line number="27" hits="1" branch="False" />
<line number="28" hits="1" branch="False" />
<line number="29" hits="1" branch="False" />
<line number="30" hits="1" branch="False" />
<line number="33" hits="1" branch="False" />
<line number="34" hits="1" branch="False" />
<line number="35" hits="1" branch="True" condition-coverage="100% (2/2)">
<conditions>
<condition number="0" type="jump" coverage="100%" />
</conditions>
</line>
<line number="36" hits="1" branch="False" />
<line number="37" hits="1" branch="False" />
</lines>
</class>
</classes>
</package>
<package line-rate="1" branch-rate="1" complexity="10" name="Algorithms.Core">
<classes>
<class line-rate="1" branch-rate="1" complexity="10" name="Algorithms.Core.Merger" filename="/home/runner/work/codecoverage/codecoverage/samples/Algorithms/src/Algorithms.Core/Merger.cs">
Expand Down
Binary file modified samples/Algorithms/scenarios/scenario01/example.report.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified samples/Algorithms/scenarios/scenario02/azure-pipelines.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -1,74 +1,6 @@
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<coverage line-rate="1" branch-rate="1" complexity="15" version="1.9" timestamp="1704372866" lines-covered="34" lines-valid="34" branches-covered="12" branches-valid="12">
<coverage line-rate="1" branch-rate="1" complexity="10" version="1.9" timestamp="1705670663" lines-covered="17" lines-valid="17" branches-covered="10" branches-valid="10">
<packages>
<package line-rate="1" branch-rate="1" complexity="5" name="Algorithms.Core.Tests">
<classes>
<class line-rate="1" branch-rate="1" complexity="5" name="Algorithms.Core.Tests.MergerTests" filename="/home/runner/work/codecoverage/codecoverage/samples/Algorithms/tests/Algorithms.Core.Tests/MergerTests.cs">
<methods>
<method line-rate="1" branch-rate="1" complexity="1" name="Test1" signature="()">
<lines>
<line number="13" hits="1" branch="False" />
<line number="14" hits="1" branch="False" />
<line number="15" hits="1" branch="False" />
<line number="16" hits="1" branch="False" />
</lines>
</method>
<method line-rate="1" branch-rate="1" complexity="1" name="Test2" signature="()">
<lines>
<line number="20" hits="1" branch="False" />
<line number="21" hits="1" branch="False" />
<line number="22" hits="1" branch="False" />
<line number="23" hits="1" branch="False" />
</lines>
</method>
<method line-rate="1" branch-rate="1" complexity="1" name="Test3" signature="()">
<lines>
<line number="27" hits="1" branch="False" />
<line number="28" hits="1" branch="False" />
<line number="29" hits="1" branch="False" />
<line number="30" hits="1" branch="False" />
</lines>
</method>
<method line-rate="1" branch-rate="1" complexity="2" name="AssertArrays" signature="(int[], int[])">
<lines>
<line number="33" hits="1" branch="False" />
<line number="34" hits="1" branch="False" />
<line number="35" hits="1" branch="True" condition-coverage="100% (2/2)">
<conditions>
<condition number="0" type="jump" coverage="100%" />
</conditions>
</line>
<line number="36" hits="1" branch="False" />
<line number="37" hits="1" branch="False" />
</lines>
</method>
</methods>
<lines>
<line number="13" hits="1" branch="False" />
<line number="14" hits="1" branch="False" />
<line number="15" hits="1" branch="False" />
<line number="16" hits="1" branch="False" />
<line number="20" hits="1" branch="False" />
<line number="21" hits="1" branch="False" />
<line number="22" hits="1" branch="False" />
<line number="23" hits="1" branch="False" />
<line number="27" hits="1" branch="False" />
<line number="28" hits="1" branch="False" />
<line number="29" hits="1" branch="False" />
<line number="30" hits="1" branch="False" />
<line number="33" hits="1" branch="False" />
<line number="34" hits="1" branch="False" />
<line number="35" hits="1" branch="True" condition-coverage="100% (2/2)">
<conditions>
<condition number="0" type="jump" coverage="100%" />
</conditions>
</line>
<line number="36" hits="1" branch="False" />
<line number="37" hits="1" branch="False" />
</lines>
</class>
</classes>
</package>
<package line-rate="1" branch-rate="1" complexity="10" name="Algorithms.Core">
<classes>
<class line-rate="1" branch-rate="1" complexity="10" name="Algorithms.Core.Merger" filename="/home/runner/work/codecoverage/codecoverage/samples/Algorithms/src/Algorithms.Core/Merger.cs">
Expand Down
Binary file modified samples/Algorithms/scenarios/scenario02/example.report.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified samples/Algorithms/scenarios/scenario03/azure-pipelines.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<coverage line-rate="1" branch-rate="1" complexity="5" version="1.9" timestamp="1704439252" lines-covered="34" lines-valid="34">
<coverage line-rate="1" branch-rate="1" complexity="1" version="1.9" timestamp="1705670665" lines-covered="17" lines-valid="17">
<packages>
<package line-rate="1" branch-rate="1" complexity="1" name="Algorithms.Core">
<classes>
Expand Down Expand Up @@ -49,65 +49,5 @@
</class>
</classes>
</package>
<package line-rate="1" branch-rate="1" complexity="4" name="Algorithms.Core.Tests">
<classes>
<class line-rate="1" branch-rate="1" complexity="4" name="Algorithms.Core.Tests.MergerTests" filename="/home/runner/work/codecoverage/codecoverage/samples/Algorithms/tests/Algorithms.Core.Tests/MergerTests.cs">
<methods>
<method line-rate="1" branch-rate="1" complexity="1" name="Test1" signature="()">
<lines>
<line number="13" hits="1" branch="False" />
<line number="14" hits="1" branch="False" />
<line number="15" hits="1" branch="False" />
<line number="16" hits="1" branch="False" />
</lines>
</method>
<method line-rate="1" branch-rate="1" complexity="1" name="Test2" signature="()">
<lines>
<line number="20" hits="1" branch="False" />
<line number="21" hits="1" branch="False" />
<line number="22" hits="1" branch="False" />
<line number="23" hits="1" branch="False" />
</lines>
</method>
<method line-rate="1" branch-rate="1" complexity="1" name="Test3" signature="()">
<lines>
<line number="27" hits="1" branch="False" />
<line number="28" hits="1" branch="False" />
<line number="29" hits="1" branch="False" />
<line number="30" hits="1" branch="False" />
</lines>
</method>
<method line-rate="1" branch-rate="1" complexity="1" name="AssertArrays" signature="(int[], int[])">
<lines>
<line number="33" hits="1" branch="False" />
<line number="34" hits="1" branch="False" />
<line number="35" hits="1" branch="False" />
<line number="36" hits="1" branch="False" />
<line number="37" hits="1" branch="False" />
</lines>
</method>
</methods>
<lines>
<line number="13" hits="1" branch="False" />
<line number="14" hits="1" branch="False" />
<line number="15" hits="1" branch="False" />
<line number="16" hits="1" branch="False" />
<line number="20" hits="1" branch="False" />
<line number="21" hits="1" branch="False" />
<line number="22" hits="1" branch="False" />
<line number="23" hits="1" branch="False" />
<line number="27" hits="1" branch="False" />
<line number="28" hits="1" branch="False" />
<line number="29" hits="1" branch="False" />
<line number="30" hits="1" branch="False" />
<line number="33" hits="1" branch="False" />
<line number="34" hits="1" branch="False" />
<line number="35" hits="1" branch="False" />
<line number="36" hits="1" branch="False" />
<line number="37" hits="1" branch="False" />
</lines>
</class>
</classes>
</package>
</packages>
</coverage>
Binary file modified samples/Algorithms/scenarios/scenario03/example.report.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
106 changes: 106 additions & 0 deletions samples/Algorithms/scenarios/scenario04/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# Scenario Description

Collect code coverage using `dotnet-coverage` tool for MSTest runner project.

> **_NOTE:_** MSTest runner project coverage extension by default is not collecting native code coverage. If you want to enable please set to `True` `EnableStaticNativeInstrumentation` or `EnableDynamicNativeInstrumentation` in configuration.
# Collect code coverage using command line

```shell
git clone https://github.com/microsoft/codecoverage.git
cd codecoverage/samples/Algorithms/tests/Algorithms.Core.Tests/
dotnet build
dotnet tool install -g dotnet-coverage
dotnet-coverage collect --output report.cobertura.xml --output-format cobertura "dotnet run --no-build"
```

You can also use [run.ps1](run.ps1) to collect code coverage.

# Collect code coverage inside github workflow

`reportgenerator` can be used to generate final github summary markdown.

```yml
steps:
- uses: actions/checkout@v3
- name: Setup .NET
uses: actions/setup-dotnet@v3
with:
dotnet-version: 8.0.x
- name: Restore dependencies
run: dotnet restore
- name: Build
run: dotnet build --no-restore
- name: Install dotnet-coverage
run: dotnet tool install -g dotnet-coverage
- name: Test
run: dotnet-coverage collect --output $GITHUB_WORKSPACE/report.cobertura.xml --output-format cobertura "dotnet run --no-build"
- name: ReportGenerator
uses: danielpalme/[email protected]
with:
reports: '${{ github.workspace }}/report.cobertura.xml'
targetdir: '${{ github.workspace }}/coveragereport'
reporttypes: 'MarkdownSummaryGithub'
- name: Upload coverage into summary
run: cat $GITHUB_WORKSPACE/coveragereport/SummaryGithub.md >> $GITHUB_STEP_SUMMARY
- name: Archive code coverage results
uses: actions/upload-artifact@v3
with:
name: code-coverage-report
path: ${{ github.workspace }}/report.cobertura.xml
```
[Full source example](../../../../.github/workflows/Algorithms_Scenario04.yml)
[Run example](../../../../../../actions/workflows/Algorithms_Scenario04.yml)
# Collect code coverage inside Azure DevOps Pipelines
```yml
steps:
- task: DotNetCoreCLI@2
inputs:
command: 'restore'
projects: '$(projectPath)' # this is specific to example - in most cases not needed
displayName: 'restore'

- task: DotNetCoreCLI@2
inputs:
command: 'build'
arguments: '--no-restore --configuration $(buildConfiguration)'
projects: '$(projectPath)' # this is specific to example - in most cases not needed
displayName: 'build'

- task: DotNetCoreCLI@2
inputs:
command: 'custom'
custom: "tool"
arguments: 'install -g dotnet-coverage'
displayName: 'install dotnet-coverage'

- task: Bash@3
inputs:
targetType: 'inline'
script: 'dotnet-coverage collect --output-format cobertura --output $(Agent.TempDirectory)/report.cobertura.xml "dotnet run --project $(projectPath) --no-build --report-trx --configuration $(buildConfiguration) --results-directory $(Agent.TempDirectory)"'
displayName: 'test'

- task: PublishTestResults@2
inputs:
testResultsFormat: 'VSTest'
testResultsFiles: '$(Agent.TempDirectory)/**/*.trx'
publishRunAttachments: false

- task: PublishCodeCoverageResults@2
inputs:
summaryFileLocation: $(Agent.TempDirectory)/**/*.cobertura.xml
```
[Full source example](azure-pipelines.yml)
![alt text](azure-pipelines.jpg "Code Coverage tab in Azure DevOps pipelines")
# Report example
![alt text](example.report.jpg "Example report")
[Link](example.report.cobertura.xml)
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit b853d98

Please sign in to comment.