Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate post-fit sampling for the cases with high statistics and large lnN uncertainties #297

Open
anigamova opened this issue Mar 10, 2023 · 3 comments

Comments

@anigamova
Copy link
Collaborator

Post-fit uncertainty bands are not correctly evaluated from the cov. matrix sampling for bins with high statistics and large lnN uncertainties, reported at CMSTalk.

@anigamova anigamova converted this from a draft issue Mar 10, 2023
@kcormi
Copy link
Collaborator

kcormi commented Mar 15, 2023

This same issue appears for asymmetric gaussian uncertainties with high statistics as well.

As suggested by others, one thing we may want to look into is using the 68% quantile of the sampled toys rather than the variance directly in the computation. I think there may be some other underlying issues in this case that might not be solved, but in general using the asymmetric interval might be a useful change.

@ajgilbert
Copy link
Collaborator

See nice summary from @nsmith- here: https://cms-talk.web.cern.ch/t/postfit-uncertainty-bands-very-large/20967/25
Though it doesn't fix the problem, we should align the calculation in CH with the one in FitDiagnostics (subtract from mean instead of best-fit values).

@anigamova
Copy link
Collaborator Author

Thanks @ajgilbert
Would this work #298 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

No branches or pull requests

3 participants