Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multiple collectors #204

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Multiple collectors #204

wants to merge 4 commits into from

Commits on Nov 10, 2020

  1. Avoid opening too many jobs in the collector. Only parse so many at o… 

    …ne time.

Most unix systems limit the number of file handles any one process can
have open at a time to 1024. When the collector gets behind, it may try
to open too many files for completed jobs at once. Limit this with a
new collector setting which defaults to 300. They'll be picked up once
the ones polled are processed.

This commit adds App::Yath::Options::Collector
    @toddr
    toddr committed Nov 10, 2020
    Configuration menu
    Copy the full SHA
    83c92da View commit details
    Browse the repository at this point in the history

  2. Clean up and re-work PR for too many open jobs 

     * Added another collector option
 * Renamed collector option
 * Put option in test and start commands
 * Make warning when job limit is hit a proper event
 * Only issue the warning once per test run
 * Use the correct settings file in collector
    @exodist
    exodist committed Nov 10, 2020
    Configuration menu
    Copy the full SHA
    e81da2d View commit details
    Browse the repository at this point in the history

  3. Fix minor bug in collector 

    Primarily a typo, but also a race condition that could have collector
exit with jobs still waiting to send events.
    @exodist
    exodist committed Nov 10, 2020
    Configuration menu
    Copy the full SHA
    923dfd2 View commit details
    Browse the repository at this point in the history

  4. Add option to allow multiple collectors 

    I am too tired right now after finally getting this working. It needs
review, and I will probably want to clean it up myself after some sleep.
    @exodist
    exodist committed Nov 10, 2020
    Configuration menu
    Copy the full SHA
    4734dd2 View commit details
    Browse the repository at this point in the history