Failure in misfit preprocessor after warning

[larsevj]

What happened? (You can include a screenshot if it helps explain)

https://fmu-dashboard-fdeqekb8chekcqhu.neu.grafana.azure.com/d/ae0bpm0169ou8e/traces?from=now-24h&to=now&var-operation_id=9a8b920d1f656a017b3f4e6aa26e1925

User report where ERT failed in the update step due to an error in misfit preprocessor. The following warning was emitted:

/prog/res/komodo/2024.11.02-py311-rhel8/root/lib64/python3.11/site-packages/ert/analysis/misfit_preprocessor.py:65: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined. correlation = spearmanr(responses).statistic.

and then ERT fails afterwards with:

error_msg='y must be 1 or 2 dimensional.

What did you expect to happen?

For ERT to either skip the auto scaling with a warning, or for ERT to error with a reasonable error message.

steps to reproduce

Not sure how exactly it was produced by the user, but can "reproduce" it by giving a constant matrix as first input arg to `cluster_responses` in misfit_preprocessor.py, then scipy.stats.spearmanr will produce nans after emitting the warning above, and we do not check for this case.

See: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.spearmanr.html

Environment where bug has been observed

• python 3.10
• python 3.11
• python 3.12
• macosx
• rhel7
• rhel8
• local queue
• lsf queue
• slurm queue
• openPBS queue

[eivindjahren]

@larsevj Until we know more about the failure, and how to reproduce it, we can't really get going nor triage this bug. I see that you were going to write more about it. Did you have anything more to write?

[xjules]

Let's add stacktrace to the exception to get more info on the error. @larsevj will write more on this.