Update mutation-testing.md #46064
Open
Update mutation-testing.md #46064
+43
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added section on how to interpret Mutation Testing Results
dotnet-policy-service
bot
added
dotnet-fundamentals/svc
community-contribution
Removed spaces before the list items from interpreting mutation results
Added some more information about CI/CD integration and custom configuration of the tool
|
@dotnet-policy-service agree |
Removed myself from the authors since I saw it should be only one
IEvangelist
requested changes
May 29, 2025
| @@ -110,6 +110,49 @@ Stryker supports several types of mutations: | |||
|
|
|||
| For additional mutation types, see the [Stryker.NET: Mutations](https://stryker-mutator.io/docs/stryker-net/mutations) documentation. | |||
|
|
|||
| ## Interpreting Mutation Testing Results | |||
There was a problem hiding this comment.
Suggested change
| ## Interpreting Mutation Testing Results | |
| ## Interpret mutation testing results |
| @@ -110,6 +110,49 @@ Stryker supports several types of mutations: | |||
|
|
|||
| For additional mutation types, see the [Stryker.NET: Mutations](https://stryker-mutator.io/docs/stryker-net/mutations) documentation. | |||
|
|
|||
| ## Interpreting Mutation Testing Results | |||
|
|
|||
| After running Stryker.NET, you’ll receive a report that categorizes mutants as **killed**, **survived**, or **timeout**. Here's how to interpret and act on these results: | |||
There was a problem hiding this comment.
Suggested change
| After running Stryker.NET, you’ll receive a report that categorizes mutants as **killed**, **survived**, or **timeout**. Here's how to interpret and act on these results: | |
| After running Stryker.NET, you'll receive a report that categorizes mutants as **killed**, **survived**, or **timeout**. Here's how to interpret and act on these results: |
|
|
||
| After running Stryker.NET, you’ll receive a report that categorizes mutants as **killed**, **survived**, or **timeout**. Here's how to interpret and act on these results: | ||
|
|
||
| - **Killed Mutants**: These are changes that your tests successfully caught. A high number of killed mutants indicates that your test suite effectively detects logic errors. |
There was a problem hiding this comment.
Suggested change
| - **Killed Mutants**: These are changes that your tests successfully caught. A high number of killed mutants indicates that your test suite effectively detects logic errors. | |
| - **Killed**: These are changes that your tests successfully caught. A high number of killed mutants indicates that your test suite effectively detects logic errors. |
|
|
||
| - **Killed Mutants**: These are changes that your tests successfully caught. A high number of killed mutants indicates that your test suite effectively detects logic errors. | ||
|
|
||
| - **Survived Mutants**: These changes were not caught by your tests. Review them to identify gaps in test coverage or assertions that are too weak. Focus on adding targeted unit tests that would fail if the mutant were real. |
There was a problem hiding this comment.
Suggested change
| - **Survived Mutants**: These changes were not caught by your tests. Review them to identify gaps in test coverage or assertions that are too weak. Focus on adding targeted unit tests that would fail if the mutant were real. | |
| - **Survived**: These changes weren't caught by your tests. Review them to identify gaps in test coverage or assertions that are too weak. Focus on adding targeted unit tests that would fail if the mutant were real. |
|
|
||
| - **Survived Mutants**: These changes were not caught by your tests. Review them to identify gaps in test coverage or assertions that are too weak. Focus on adding targeted unit tests that would fail if the mutant were real. | ||
|
|
||
| - **Timeout Mutants**: These mutations caused your code to hang or exceed the allowed time. This can happen with infinite loops or unoptimized paths. Investigate the code logic or increase the timeout threshold if needed. |
There was a problem hiding this comment.
Suggested change
| - **Timeout Mutants**: These mutations caused your code to hang or exceed the allowed time. This can happen with infinite loops or unoptimized paths. Investigate the code logic or increase the timeout threshold if needed. | |
| - **Timeout**: These mutations caused your code to hang or exceed the allowed time. This can happen with infinite loops or unoptimized paths. Investigate the code logic or increase the timeout threshold if needed. |
| Besides setting thresholds for your pipeline, Stryker.NET offers the possibility of having different configurations for each of your project needs. You can do this customisation of behaviour using the stryker-config.json file. | ||
| ``` | ||
| { | ||
| "ignoreMutations": [ |
There was a problem hiding this comment.
Why is the naming convention of the config keys different? In one case we see ignoreMutations and in the other we see ignore-methods? Are both acceptable?
Comment on lines
+152
to
+154
| - **IgnoreMutations**: These are the methods or expressions that you want to ignore because they are noisy or you consider not needed based on your application logic needs. They will show up in your reports as Ignored. | ||
| - **Ignore-methods**: You can use this to skip entire methods based on their signatures. Also, those will appear in your reports as Ignored. In our example, we ignore all methods ending in "Logs". | ||
| - **Mutate**: Without this option, Styker will try to mutate all the files in your project. With this, you can ignore files or entire folders. In the example above, we ignore everything inside a Migrations folder and all .Designer.cs files (which are usually autogenerated) |
There was a problem hiding this comment.
Suggested change
| - **IgnoreMutations**: These are the methods or expressions that you want to ignore because they are noisy or you consider not needed based on your application logic needs. They will show up in your reports as Ignored. | |
| - **Ignore-methods**: You can use this to skip entire methods based on their signatures. Also, those will appear in your reports as Ignored. In our example, we ignore all methods ending in "Logs". | |
| - **Mutate**: Without this option, Styker will try to mutate all the files in your project. With this, you can ignore files or entire folders. In the example above, we ignore everything inside a Migrations folder and all .Designer.cs files (which are usually autogenerated) | |
| - **ignore-mutations**: These are the methods or expressions that you want to ignore because they are noisy, or you consider not needed based on your application logic needs. They will show up in your reports as `Ignored`. | |
| - **ignore-methods**: You can use this to skip entire methods based on their signatures. Also, those will appear in your reports as Ignored. In our example, we ignore all methods ending in "Logs". | |
| - **mutate**: Without this option, Styker will try to mutate all the files in your project. With this, you can ignore files or entire folders. In the example above, we ignore everything inside a _Migrations_ folder and all _.Designer.cs_ files (which are usually autogenerated) . |
| Besides setting thresholds for your pipeline, Stryker.NET offers the possibility of having different configurations for each of your project needs. You can do this customisation of behaviour using the stryker-config.json file. | ||
| ``` | ||
| { | ||
| "ignoreMutations": [ |
There was a problem hiding this comment.
Suggested change
| "ignoreMutations": [ | |
| "ignore-mutations": [ |
| - **IgnoreMutations**: These are the methods or expressions that you want to ignore because they are noisy or you consider not needed based on your application logic needs. They will show up in your reports as Ignored. | ||
| - **Ignore-methods**: You can use this to skip entire methods based on their signatures. Also, those will appear in your reports as Ignored. In our example, we ignore all methods ending in "Logs". | ||
| - **Mutate**: Without this option, Styker will try to mutate all the files in your project. With this, you can ignore files or entire folders. In the example above, we ignore everything inside a Migrations folder and all .Designer.cs files (which are usually autogenerated) | ||
|
|
There was a problem hiding this comment.
Suggested change
| For more information, see [Stryker: Configuration](https://stryker-mutator.io/docs/stryker-net/configuration/). | |
Comment on lines
+134
to
+135
| ## Costumization | ||
| Besides setting thresholds for your pipeline, Stryker.NET offers the possibility of having different configurations for each of your project needs. You can do this customisation of behaviour using the stryker-config.json file. |
There was a problem hiding this comment.
Suggested change
| ## Costumization | |
| Besides setting thresholds for your pipeline, Stryker.NET offers the possibility of having different configurations for each of your project needs. You can do this customisation of behaviour using the stryker-config.json file. | |
| ## Customization | |
| Besides setting thresholds for your pipeline, Stryker.NET offers the possibility of having different configurations for each of your project needs. You can do this customization of behavior using the _stryker-config.json_ file. |
IEvangelist
reviewed
May 29, 2025
Comment on lines
+127
to
+133
| ``` | ||
| "thresholds": { | ||
| "high": 85, | ||
| "low": 65, | ||
| "break": 0 | ||
| } | ||
| ``` |
There was a problem hiding this comment.
Why is this here? There's no mention of config yet, it's just a code block with values that are not described anywhere. Let's remove this.
IEvangelist
reviewed
May 29, 2025
| ``` | ||
| ## Costumization | ||
| Besides setting thresholds for your pipeline, Stryker.NET offers the possibility of having different configurations for each of your project needs. You can do this customisation of behaviour using the stryker-config.json file. | ||
| ``` |
There was a problem hiding this comment.
Suggested change
| ``` | |
| ```json |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Improved documentation
Summary
Describe your changes here.
Fixes #Issue_Number (if available)
Internal previews