GUNDAM v1.8.6 Patch Release

This patch release marks the state of GUNDAM after the completion of T2K validation against previous near detector fits.

Fixes relative to 1.8.5

Fix (Merge #681): Handle over sized graphs and splines. The number of points that can be handled in a spline/graph has been increased, but that needs to be used carefully since large splines and graphs will make fits run more slowly. Note: Using linear interpolation with more than two knots is explicitly forbidden by MINUIT since it introduces a discontinuous derivative. MINUIT may still run, but the results are undefined (that means the answer could be useful, or could start a thermonuclear war. Both outcomes are formally correct).

Fix (Merge #676 and #665): Make usage of GTest unit testing optional. The full testing suite can be disabled by setting the CMake option ENABLE_TESTS to OFF. Just the unit tests can be disabled by setting SKIP_GOOGLE_TESTS to ON. Warning messages during the CMake build are now actual warnings that should be noticed by the users.

Issue #674: Change termination behavior when DialInputBuffer finds an invalid parameter. Previously it would throw, and it has been changed to print a descriptive message and the exit with failure. This is because the most common way that there will be a bad parameter value in DialInputBuffer is from a bad user configuration file (or other user inputs).

Issue #673: Fix linear interpolation when there are more than two knots. Interpolation for two knots was working correctly.

Issue #615: Fix discontinuities in the JointProbability for the MC and data in a histogram bin. The joint probability calculation has to be correct over ~620 powers of ten, so there is a lot of room for numeric issues. The probability is cleaned up so that it has all of the right limits, and "never" fails numerically (never means we haven't found anymore corner cases). The new implementation numerically matches the OA2021 when the OA2021 implementation is valid, and is smooth outside of that domain. Options are added to reproduce the old bugs.

Fix (Merge #620): Fix an infinite loop when parameters are being drawn from a global convariance.

Issue #621: GPU normalization weight was not being checked for validity before being applied to the event weight. This only caused problems in corner cases, but when it caused trouble, you could end up with negative expectations. It was found while torture testing the detector systematics. The normalization now handles invalid inputs.

Full Changelog: 1.8.5...1.8.6

GUNDAM v1.8.5 Patch Release

This patch concentrates on making sure that the parameter values always honor the user set minimum and maximum bounds. It also improves checks that the input configuration files are describing a valid likelihood, and that the likelihood definition is not changed during the fit.

Fixes relative to 1.8.4

Issue #582: Safely check parameter limits during fits. The old method for checking parameter validity during fits depended on first setting an invalid value, and then checking the validity. Queries are added to check validity before setting. This also adds checks for the GPU that the likelihood has not been changed since initialization. If there is a change, a log message is generated, and the calculation falls back to the CPU. This should only happen during debugging.

Issue #578: Apply cleanups to make sure that eigen decomposition and parameter throwing honor the parameter boundaries.

Fixes: Guarantee that all parameter value access is through the setter and getter so that the validity checks are correctly applied.

Full Changelog: 1.8.4...1.8.5

GUNDAM v1.8.4 Patch Release

This patch includes fixes to : 1) substantially increase the efficiency of summing the histograms when using the GPU; 2) Add validation the reweighting and check that caching used by CPU calculation correctly updates; and, 3) add a collection of quality of life fixes (particularly, fewer warnings during compilation).

Fixes relative to 1.8.3

Issue #545 : Make the handling of ROOT config information portable between linux and macos. This no longer assumes how ROOT is installed, and uses root-config safely.

Update (Related to #540): Strengthen the NaN trap in the JointProbability calculation. An NaN there means the fit cannot continue, so throw an error. Continue to allow INF.

Update (Prompted by #541): Make compilation less alarming by quieting warnings. They were originally part of debugging.

Issue #536 : Add control over the size of a "kick" when starting a fit. The starting point can be fluctuated around the prior, and the size of the step is controlled by an optional argument to --kickmc

Issue #534 : Backport CPU and GPU calculation backing from main, and add validation that the two calculations agree within machine precision.

Issue #530 : Fix an inefficiency in the summation of the event weights on the GPU and double the speed of the likelihood calculation using the GPU.

Issue # 524 : Apply the global event weight cap when using the Cache::Manager to calculate the likelihood.

Full Changelog: 1.8.3...1.8.4

GUNDAM v1.8.3 Patch Release

This patch includes fixes to : 1) Stop output of disabled parameters; 2) fix parameters using the configuration file; 3) reduce the number of compilation warnings with CUDA; and, 4) substantially increase the efficiency with a GPU.

Fixes relative to 1.8.2

Associated Fix: The CalculateGeneralSpline and CalculateGraph functions have been changed to use a brute-force binary search in place of the linear search for the correct index. This results in a speed up of about 50% for those routines.

Issue #510 : Make the CacheManager calculation much more thread save when being run on the CPU. This isn't the normal mode, but is an important cross check. The CPU now uses (lock free) atomic addition, multiplication and value setting.

Issue #513 : Fix JointProbability so that an infinite log likelihood produces a warning, but does not terminate the program. Infinities are valid (e.g. a zero probability). This does produce a warning since they shouldn't occur often during a normal run.

Issue #506 : Fix output statement in ParameterSet.cpp to remove duplicated output.

Issue #505 : Add the "isFixed" option for parameters in the configuration file.

Issue #508 : Improve error checking for Cache::Manager. This makes sure that LogThrow is preferred to std::runtime_error. There is an issue filed for simple-cpp-logger to make sure that the output is flushed before throwing, so that should make the error output much more readable.

Issue #502 : Update the validation code. A validation failure caused by the recent corrections to the monotonic spline calculation (#486 and #494) has been fixed. The expected result has been updated. Unit testing with GoogleTest has been added and used for the HEMI GPU interface code. Further tests are being implemented.

Issue #492 : Fix compiler warnings from NVCC. This is removing some unused variables. It also fixes a "fix" that applied some correct C++ conventions to code that is aimed at the GPU (the fix produces inefficient code on a SIMD processor, and was primarily aesthetic.

Issue #490 : Don't set the value for disabled parameters in the output histograms. This will prevent uninitialized values (i.e. NAN) from appearing in plots.

Full Changelog: 1.8.2...1.8.3

GUNDAM V1.8.2 Patch Release

Fixes relative to 1.8.1

Issue #499 : Update to a new version of the cpp-generic-toolkit submodule. The cpp-generic-toolkit fix sets the branch status to enable branches that are used for the data.

Issue #494 : Rearrange the application of the Fritschle-Carlson criteria so that it can be tested. This also applies a fix to Fritschle-Carlson so that the end of the spline are handled in a more reasonable fashion.

Issue #493 : Add tests for ComputeGeneralSplines to the validation suite. As a special bonus it also adds tests for CalculateUniformSpline.h, and resolves unused variable warnings for both those functions.

Summary

• Resolve LTS 1.8.x issue493: Add general spline test by @ClarkMcGrew in #495
• Resolve LTS/1.8.x issue 494: Check spline monotonicity by @ClarkMcGrew in #496
• Resolve LTS/Issue499: Set the status of branches by @ClarkMcGrew in #500

Full Changelog: 1.8.1...1.8.2

GUNDAM v1.8.1 Patch Release

Fixes relative to 1.8.0

Issue #485 : A job will correctly continue when the likelihood returns an infinite value. The job will stop if the likelihood returns a NaN.

Issue #486 : Fix the Catmull-Rom splines so that they have symmetric behavior. This changes the extrapolation behavior for both Catmull-Rom and Catmull-Rom,monotonic splines. The Catmull-Rom monotonic splines are updated to use the full Fritsche-Carlson criteria so that the interpolation is smoother.

What's Changed

• Fix GUNDAM 1.8.x Issue #486: Catmull Rom Extrapolation by @ClarkMcGrew in #487
• GUNDAM 1.8.0: Fix for issue #485: LLH is throwing when there is a Inf (should continue) by @ClarkMcGrew in #488

Full Changelog: 1.8.0...1.8.1

GUNDAM v1.9.0

This is a development, and cannot be considered validated

Improving class structure and better defining herarchy and ownership (see figure bellow). This is primarily a development release and will not receive long term support.

As usual, backward compatibility of config files is ensured! This version ended up around twice faster than the previous one. 😄

Additions

• Adding multiparameter dials
• Adding Formula and precompiled dials
• Adding app to compare fit results (sensitivity studies for instance): gundamFitPlot
• Not throwing an error when a NaN values is propagated in the reweight pipeline.
  • Migrads might explore these regions while adjusting the step size
• Using ordered JSON unfolding
• CMake option DISABLE_ZLIB to compile without Zlib (this library is only used to make the propagator cache faster)

Fixes

• Fixed gundamRoot with latest versions of ROOT
• Fixing event-by-event dial loading with multiple datasets
• update.sh script now properly updates the submodules

Full changelog of 1.8.0-> 1.9.0

Gundam v1.8.0

The 1.8 series has been designated for long term support, and will be patched as bugs are found, but new feature are going to be avoided.

Additions

• Every event variable can now be custom defined thanks to variableDict
• Adding dialIndexFormula in order to allow the user to select a given entry of a TClonesArray for event-by-event dials
• Adding branch commit number to the displayed version displayed and stored by GUNDAM apps
• Adding globalEventReweightCap to apply a maximum value to the weight an event could get.
• Adding commit hash + git branch on GUNDAM version printout when not on tagged version
• gundamCalcXsec can now be ran on priors covariance matrix
• gundamCalcXsec support arbitrary binning
• Better multithread support with proper signal wait
• Changed pbar to display GUNDAM intead of T2K since it is beeing used outside of T2K
• Additional prinouts for breaking down parameters and sample before the fit

Fixes

• Various misleading printouts
• Better CMake support
• Proper handling of nan values in JSON parsing for old versions
• Optimized data loader

Full changelog of 1.7.2-> 1.8.0

Gundam v1.7.2

Reintroduction of the CalcXsec!

• gundamCalcXsec is now working with the new dial/cache system of the propagator introduced since 1.7.0
• Now supporting parameter dependent renomalizations (like flux renormalisation)
• Don't load the fit samples in the xsection calc propagator (way faster)
• Use config override feature to tweak the parameter definitions used during the fit

Fixes

• CacheManager is now running with the new dial/cache system (i.e. CUDA is back!)
• Now working with ROOT v6.28.04+
• Now cmake checks if the submodules are present
• Full synced event/dial cache sorting to insure reproducibility of toy throws and fit runs

Additions

• ./update.sh is a magical script that will help you update the code and its submodule effortlessly 🙃
• Updating doc

/!\ Submodules have been moved to the gundam-organization for better support.

Some old version of git might freak out while updating, so if you encounter some troubles while recompiling
(GenericToolbox::... could not find function for instance), you can either try to update manually the submodules
or start back from scratch: i.e.:

git clone https://github.com/gundam-organization/gundam.git
cd gundam
./update.sh --latest

Full changelog of 1.7.1-> 1.7.2

Gundam v1.7.1

Fixes

• Fixing logger issue while building with C++14
• Fixing ParScan config loading ending up in a segfault

Additions

• Systematically check for GUNDAM version while building
• Adding gundamFitReader app that allows to printout the results of a fit using the fitter output file
• Adding gundamInputZipper app that allows to gather configs and resource file in a local place, and zip it. This allows to pass a completely portable set of inputs for another user who wants to run the fit.
• Adding override config feature that allows to edit a main configuration: gundamFitter -c config
• Adding LineScanner feature that allows to monitor the LLH response on a straight line in the parameter space
• Adding convergence monitor that draws the path taken by Minuit to find the minimum

Full changelog of 1.7.0-> 1.7.1