Problem Description

safe-settings supports running in a nop mode which will report the changes that will happen in a PR comment. Recently Environments support was added in #544 but this does not seem to work with the nop mode. When we opened a PR adding an environment it went and applied those changes directly before the PR was merged to master.

What is actually happening

Opening a PR adding an environment to a repo will add it when running the nop mode in the PR checks and then not report any changes in the PR comment.

What is the expected behavior

Opening a PR adding an environment should report the changes being made in the PR comment and only apply the changes when the PR is merged to master.

Error output, if available

N/A

Context

Are you using the hosted instance of probot/settings or running your own?

Running our own.

If running your own instance, are you using it with github.com or GitHub Enterprise?

github.com

Version of probot/settings

2.1.0 (latest stable)

Version of GitHub Enterprise

N/A