Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for various non-zero exit codes for cli utilities #225

Open
m-sedl opened this issue Jan 30, 2023 · 0 comments
Open

Support for various non-zero exit codes for cli utilities #225

m-sedl opened this issue Jan 30, 2023 · 0 comments

Comments

@m-sedl
Copy link
Collaborator

m-sedl commented Jan 30, 2023

Now VSharp and TestRunner will return a null code even if some of the tests fail, or if exceptions are thrown during branch parsing.

I propose to consider the possibility of returning different codes depending on the quality of the work done. For example, if most of the generated tests fail, then you can return a special code. Also, the codes may depend on the degree of importance of the detected errors.

An example of such a system can be found here: https://pvs-studio.com/en/docs/manual/0035/.

The proposed approach will make it more convenient to embed VSharp in CI systems - automation will be able to interpret the results in a single way. In particular, it will be easier for us to understand with what status the benchmarks ended.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant