Skip to content
This repository has been archived by the owner on Aug 29, 2018. It is now read-only.

cartridge control scripts: Fix exit value handling #6274

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Miciah
Copy link
Contributor

@Miciah Miciah commented Oct 14, 2015

Fix some problems with how we handle exit values in control scripts.

Most control scripts have #!/bin/bash -e or set -e so that they exit immediately if a command exits with a non-zero return value if that command is not part of a compound command or conditional statement. Thus if we anticipate that a command may fail, we must put it in a compound
command or a conditional statement. This commit makes the following changes:

• Replace foo; [ $? -eq 0 ] && bar with local rc=0; foo || rc=$?; [[ "$rc" = 0 ]] && bar where foo is an especially long command.

• Replace foo; [ $? -eq 0 ] || bar with if ! foo; then bar; fi.

• Replace foo; ret=$?; if [ $ret -eq 0 ]; then with if foo; then.

• Replace foo; ret=$?; if [ $ret -ne 0 ]; then with if ! foo; then.

• Replace foo; bar $? with local rc=0; foo || rc=$?; bar "$rc".

• Replace foo; return $? with local ret=0; foo || ret=$?; return $ret.

• If a script runs with /bin/bash -e and ends with exit $?, delete the exit $? because it is redundant.

Moreover, several control scripts had inverted logic around the /bin/kill command: These scripts would run /bin/kill, check its exit value, and retry /bin/kill if the first attempt returned 0. However, an exit value of 0 from /bin/kill means that it succeeded, so this code was retrying exactly when it did not need to do so. This commit fixes this logic.

@tiwillia, @dobbymoodge, does this make sense, or am I silly?

@tiwillia
Copy link
Member

@Miciah this all makes sense. Can't find any issues with your changes. LGTM.

Thanks for fixing this.

@Miciah
Copy link
Contributor Author

Miciah commented Oct 14, 2015

Thanks, Tim!

@VojtechVitek, would you be able to double-check this PR? I saw your name on some related changes in the Git log.

@VojtechVitek
Copy link
Contributor

LGTM

// Disclaimer: I'm ex Red Hatter :)

echo "Restarting PHP ${OPENSHIFT_PHP_VERSION} cartridge (Apache+mod_php)"
ensure_httpd_restart_succeed "$HTTPD_PID_FILE" "$HTTPD_CFG_FILE"
if [ -f "$HTTPD_PID_FILE" ]; then
ensure_httpd_restart_succeed "$HTTPD_PID_FILE" "$HTTPD_CFG_FILE" || ret=$?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, maybe we expect this always to succeed and would rather the script fail loudly if it does not.

Fix some problems with how we handle exit values in control scripts.

Most control scripts have `#!/bin/bash -e` or `set -e` so that they exit
immediately if a command exits with a non-zero return value if that
command is not part of a compound command or conditional statement.
Thus if we anticipate that a command may fail, we must put it in a compound
command or a conditional statement.  This commit makes the following
changes:

• Replace `foo; [ $? -eq 0 ] && bar` with `local rc=0; foo || rc=$?;
  [[ "$rc" = 0 ]] && bar` where foo is an especially long command.

• Replace `foo; [ $? -eq 0 ] || bar` with `if ! foo; then bar; fi`.

• Replace `foo; ret=$?; if [ $ret -eq 0 ]; then` with `if foo; then`.

• Replace `foo; ret=$?; if [ $ret -ne 0 ]; then` with `if ! foo; then`.

• Replace `foo; bar $?` with `local rc=0; foo || rc=$?; bar "$rc"`.

• Replace `foo; return $?` with `local ret=0; foo || ret=$?; return
  $ret`.

• If a script runs with `/bin/bash -e` and ends with `exit $?`, delete
  the `exit $?` because it is redundant.

Moreover, several control scripts had inverted logic around the
/bin/kill command: These scripts would run /bin/kill, check its exit
value, and retry /bin/kill if the first attempt returned 0.  However,
an exit value of 0 from /bin/kill means that it succeeded, so this code
was retrying exactly when it did not need to do so.  This commit fixes
this logic.
@Miciah Miciah force-pushed the cartridge-control-scripts-fix-exit-value-handling branch from a20fb19 to b6765ea Compare October 26, 2015 17:19
@Miciah
Copy link
Contributor Author

Miciah commented Oct 26, 2015

Sorry for pinging you, Vojtech, and thanks for graciously providing a review!

I updated the PR not to modify cartridges/openshift-origin-cartridge-php/bin/control to catch failures from ensure_httpd_restart_succeed. A failure in ensure_httpd_restart_succeed will cause the control script to terminate, and this PR now does not change this behaviour.

[test] [extended:cartridge,gear]

@Miciah
Copy link
Contributor Author

Miciah commented Oct 26, 2015

Blocked by failures in the gear extended tests, which will be fixed by #6291.

@Miciah
Copy link
Contributor Author

Miciah commented Oct 27, 2015

[test] again, please!

@Miciah
Copy link
Contributor Author

Miciah commented Oct 27, 2015

Failed because of EC2 timeouts. [test] again, please!

@tiwillia
Copy link
Member

Lets try to [test] again, its been awhile.

@tiwillia
Copy link
Member

Cartridge group 4 tests appear to be failing:

          -----------------------------------------------------------
                      Begin Output From Extended Cartridge Group 4 Tests
          -----------------------------------------------------------

# cucumber --strict -f progress -f rerun --out /tmp/rerun_1323386c76039e586972e39cf087fa5f.txt  -f junit --out /tmp/rhc/cucumber_results -t @cartridge_extended4 -t ~@not-rhel -t ~@not-origin -r openshift-test/tests @/tmp/rerun_8daaa4fc331214048e6a46f632ed4310.txt

Using RSpec 2
simple: 19308 ruby 1.9.3 2013-11-22 [x86_64-linux]
--------------------DEBUG: Using config file /root/.openshift/express.conf
DEBUG: Git config 'git config --get rhc.app-id' returned ''
DEBUG: Git config 'git config --get rhc.app-name' returned ''
DEBUG: Git config 'git config --get rhc.domain-name' returned ''
DEBUG: Authenticating with RHC::Auth::Basic
DEBUG: Connecting to https://ec2-52-23-220-226.compute-1.amazonaws.com/broker/rest/api
DEBUG: Getting all domains
DEBUG: Client supports API versions 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7
DEBUG: Created new httpclient
DEBUG: Request GET https://ec2-52-23-220-226.compute-1.amazonaws.com/broker/rest/api
DEBUG: SSL Verification failed -- Using self signed cert
DEBUG:    code 200   25 ms
DEBUG: Server supports API versions 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7
DEBUG:    Using API version 1.7
DEBUG: Client API version 1.7 is not current. Refetching API
DEBUG: Request GET https://ec2-52-23-220-226.compute-1.amazonaws.com/broker/rest/api
DEBUG:    code 200   10 ms
DEBUG: Request GET https://ec2-52-23-220-226.compute-1.amazonaws.com/broker/rest/domains
DEBUG:    code 200   16 ms
DEBUG: Request GET https://ec2-52-23-220-226.compute-1.amazonaws.com/broker/rest/domain/ci35317552/application/89999799/gear_groups
DEBUG:    code 200   78 ms
DEBUG: Using config file /root/.openshift/express.conf
DEBUG: Git config 'git config --get rhc.app-id' returned ''
DEBUG: Git config 'git config --get rhc.app-name' returned ''
DEBUG: Git config 'git config --get rhc.domain-name' returned ''
DEBUG: Authenticating with RHC::Auth::Basic
DEBUG: Connecting to https://ec2-52-23-220-226.compute-1.amazonaws.com/broker/rest/api
DEBUG: Getting all domains
DEBUG: Client supports API versions 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7
DEBUG: Created new httpclient
DEBUG: Request GET https://ec2-52-23-220-226.compute-1.amazonaws.com/broker/rest/api
DEBUG: SSL Verification failed -- Using self signed cert
DEBUG:    code 200   23 ms
DEBUG: Server supports API versions 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7
DEBUG:    Using API version 1.7
DEBUG: Client API version 1.7 is not current. Refetching API
DEBUG: Request GET https://ec2-52-23-220-226.compute-1.amazonaws.com/broker/rest/api
DEBUG:    code 200   11 ms
DEBUG: Request GET https://ec2-52-23-220-226.compute-1.amazonaws.com/broker/rest/domains
DEBUG:    code 200   20 ms
DEBUG: Request GET https://ec2-52-23-220-226.compute-1.amazonaws.com/broker/rest/domain/ci35317552/application/89999799/gear_groups
DEBUG:    code 200   77 ms
.------------------..--------------.---------------------.

(::) failed steps (::)

expected: > 0
     got:   0 (RSpec::Expectations::ExpectationNotMetError)
./openshift-test/controller/test/cucumber/step_definitions/runtime_steps.rb:353:in `/^a (.+) process will( not)? be running$/'
openshift-test/tests/cartridge-lifecycle-jboss-unified-push.feature:26:in `And a mysqld process will be running'

expected: == true
     got:    false (RSpec::Expectations::ExpectationNotMetError)
./openshift-test/controller/test/cucumber/step_definitions/cartridge-mysql-extended_steps.rb:65:in `/^the (additional )?test data will (not )?be present in mysql$/'
openshift-test/tests/cartridge-lifecycle-mysql.feature:12:in `Then the additional test data will be present in mysql'

expected: == true
     got:    false (RSpec::Expectations::ExpectationNotMetError)
./openshift-test/controller/test/cucumber/step_definitions/cartridge-mysql-extended_steps.rb:65:in `/^the (additional )?test data will (not )?be present in mysql$/'
openshift-test/tests/cartridge-lifecycle-mysql.feature:12:in `Then the additional test data will be present in mysql'

expected: == true
     got:    false (RSpec::Expectations::ExpectationNotMetError)
./openshift-test/controller/test/cucumber/step_definitions/cartridge-mysql-extended_steps.rb:65:in `/^the (additional )?test data will (not )?be present in mysql$/'
openshift-test/tests/cartridge-lifecycle-mysql.feature:42:in `Then the additional test data will be present in mysql'

expected: == true
     got:    false (RSpec::Expectations::ExpectationNotMetError)
./openshift-test/controller/test/cucumber/step_definitions/cartridge-mysql-extended_steps.rb:65:in `/^the (additional )?test data will (not )?be present in mysql$/'
openshift-test/tests/cartridge-lifecycle-mysql.feature:67:in `Then the additional test data will be present in mysql'

Failing Scenarios:
cucumber openshift-test/tests/cartridge-lifecycle-jboss-unified-push.feature:6 # Scenario: Application Creation
cucumber openshift-test/tests/cartridge-lifecycle-mysql.feature:3 # Scenario: Snapshot/Restore an application with a MySQL database
cucumber openshift-test/tests/cartridge-lifecycle-mysql.feature:3 # Scenario: Snapshot/Restore an application with a MySQL database
cucumber openshift-test/tests/cartridge-lifecycle-mysql.feature:32 # Scenario: Snapshot/Restore a scalable application with a MySQL database
cucumber openshift-test/tests/cartridge-lifecycle-mysql.feature:56 # Scenario: Snapshot/Restore between applications with a MySQL database

5 scenarios (5 failed)
91 steps (5 failed, 44 skipped, 42 passed)
9m23.490s

          -----------------------------------------------------------
                       End Output From Extended Cartridge Group 4 Tests
          -----------------------------------------------------------

@Miciah
Copy link
Contributor Author

Miciah commented Jan 12, 2016

[test] again, please!

@Miciah
Copy link
Contributor Author

Miciah commented Feb 16, 2016

Please [test] again!

@Miciah
Copy link
Contributor Author

Miciah commented Jul 27, 2016

[test] again, please!

@openshift-bot
Copy link

Evaluated for online test up to b6765ea

@openshift-bot
Copy link

Online Test Results: FAILURE (https://ci.dev.openshift.redhat.com/jenkins/job/test_pull_requests/9344/) (Extended Tests: cartridge, gear)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants