Implement new pooled mixed-mode default behavior #454

Colelyman · 2024-07-12T20:59:14Z

Now by default, when Pooled is run in mixed-mode (an amplicon file and genome are both provided) CRISPResso will demultiplex the reads that align to the locations in the genomes. This means that reads don't need to align perfectly to the same start and end coordinates. To restore the previous default behavior, one can provide the --demultiplex_genome_wide parameter.

* changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]>

CRISPResso2/CRISPRessoPooledCORE.py

kclem

Recommend undoing change at https://github.com/pinellolab/CRISPResso2/pull/454/files#diff-558d439e6beb162dcafe6859c68602f6b56597c00380fb7b625eb571f02c3c6bL388

commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: trevormartinj7 <[email protected]> * Batch d3 clean (#55) * imports C2Pro plots if available * added --use_matplotlib flag * added C2Pro matched api funciton signatures * added api args for plotly * added **kwargs * renamed config to custom_config, more specificity * added backend flag for plotly kaleido * added pro_installed boolean for templates, added plotly dependency to report templates * Squashed commit of the following: commit c909ea3b34e87ce637e00dac075d2bb2f8bfb954 Author: McKay <mbow…

commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-…

commit 354f962d4201ae4784108b0a89467796af6ec326 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:49:32 2024 -0600 specify encoding commit 823250b8108ede27b29d59d1239c74c93d74395f Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:46:17 2024 -0600 changed _ROOT to _root commit fb8cfa7614598f23e578b55b92ca9b685eef3156 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:44:32 2024 -0600 added docstrings commit e0777a805e4da9729118b429fb44687ecc15d590 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:32:34 2024 -0600 fixes for linting commit 3ade759b85992484ae59aa7db3f10fe8e6bbdc50 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:04:43 2024 -0600 Squashed commit of the following: commit 42be4bb0bcd58cdf9c902a448a7e5bde888fb1d3 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 14:36:55 2024 -0600 added pro check for plotly import in batchReport commit 31873d15fff9b9b12c1e8d8c62efa686c12cbd37 Author: mbowcut2 <[email protected]> Date: Fri Aug 16 15:05:58 2024 -0600 pointed integration tests at test branch commit d449600561074c8721322e94a8375b3ad742a88c Author: mbowcut2 <[email protected]> Date: Fri Aug 16 14:24:03 2024 -0600 Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] …

commit bb7533405af9f355ee9b367903e1f391375c7ca8 Merge: 354f962 2eff6a1 Author: mbowcut2 <[email protected]> Date: Thu Aug 22 13:47:21 2024 -0600 Merge branch 'master' into c2pro-reports commit 354f962d4201ae4784108b0a89467796af6ec326 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:49:32 2024 -0600 specify encoding commit 823250b8108ede27b29d59d1239c74c93d74395f Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:46:17 2024 -0600 changed _ROOT to _root commit fb8cfa7614598f23e578b55b92ca9b685eef3156 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:44:32 2024 -0600 added docstrings commit e0777a805e4da9729118b429fb44687ecc15d590 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:32:34 2024 -0600 fixes for linting commit 3ade759b85992484ae59aa7db3f10fe8e6bbdc50 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:04:43 2024 -0600 Squashed commit of the following: commit 42be4bb0bcd58cdf9c902a448a7e5bde888fb1d3 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 14:36:55 2024 -0600 added pro check for plotly import in batchReport commit 31873d15fff9b9b12c1e8d8c62efa686c12cbd37 Author: mbowcut2 <[email protected]> Date: Fri Aug 16 15:05:58 2024 -0600 pointed integration tests at test branch commit d449600561074c8721322e94a8375b3ad742a88c Author: mbowcut2 <[email protected]> Date: Fri Aug 16 14:24:03 2024 -0600 Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no t…

commit ae22c92e10c757667847a043eddef5a2fe333be7 Author: mbowcut2 <[email protected]> Date: Thu Aug 22 16:12:08 2024 -0600 fix cup download commit f4f8a9027a4742b883c562e31fc7fe5f45a5f8cf Merge: 2eff6a1 bb75334 Author: mbowcut2 <[email protected]> Date: Thu Aug 22 15:50:20 2024 -0600 Merge pull request #24 from edilytics/c2pro-reports C2pro reports commit bb7533405af9f355ee9b367903e1f391375c7ca8 Merge: 354f962 2eff6a1 Author: mbowcut2 <[email protected]> Date: Thu Aug 22 13:47:21 2024 -0600 Merge branch 'master' into c2pro-reports commit 354f962d4201ae4784108b0a89467796af6ec326 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:49:32 2024 -0600 specify encoding commit 823250b8108ede27b29d59d1239c74c93d74395f Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:46:17 2024 -0600 changed _ROOT to _root commit fb8cfa7614598f23e578b55b92ca9b685eef3156 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:44:32 2024 -0600 added docstrings commit e0777a805e4da9729118b429fb44687ecc15d590 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:32:34 2024 -0600 fixes for linting commit 3ade759b85992484ae59aa7db3f10fe8e6bbdc50 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 16:04:43 2024 -0600 Squashed commit of the following: commit 42be4bb0bcd58cdf9c902a448a7e5bde888fb1d3 Author: mbowcut2 <[email protected]> Date: Mon Aug 19 14:36:55 2024 -0600 added pro check for plotly import in batchReport commit 31873d15fff9b9b12c1e8d8c62efa686c12cbd37 Author: mbowcut2 <[email protected]> Date: Fri Aug 16 15:05:58 2024 -0600 pointed integration tests at test branch commit d449600561074c8721322e94a8375b3ad742a88c Author: mbowcut2 <[email protected]> Date: Fri Aug 16 14:24:03 2024 -0600 Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension…

* Fix CRISPRessoAggregate bug and other improvements (#95) * D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Add amplicon_name to plot functions * Add sgRNA sequences to nucleotide quilt parameters in Aggregate * Add custom_colors to Aggregate plot functions * Update Aggregate and make_aggregate_report to have logger and tool * Write command_used to Aggregate .json info file * Point to new test branch and add Aggregate run * Make the order of Aggregate runs explicit * Sort all instances of crispresso2_folder_info in Aggregate * Sort df_summary_quantification df in Aggregate * Try sorting with a list of single column * Update to correct test branch * Move to master test branch --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns ----…

* Fix CRISPRessoAggregate bug and other improvements (#95) * D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- * Add amplicon_name to plot functions * Add sgRNA sequences to nucleotide quilt parameters in Aggregate * Add custom_colors to Aggregate plot functions * Update Aggregate and make_aggregate_report to have logger and tool * Write command_used to Aggregate .json info file * Point to new test branch and add Aggregate run * Make the order of Aggregate runs explicit * Sort all instances of crispresso2_folder_info in Aggregate * Sort df_summary_quantification df in Aggregate * Try sorting with a list of single column * Update to correct test branch * Move to master test branch --------- * Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns ----… Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]>

* Mckay/c2pro reports test (#99) * Fix CRISPRessoAggregate bug and other improvements (#95) * D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Add amplicon_name to plot functions * Add sgRNA sequences to nucleotide quilt parameters in Aggregate * Add custom_colors to Aggregate plot functions * Update Aggregate and make_aggregate_report to have logger and tool * Write command_used to Aggregate .json info file * Point to new test branch and add Aggregate run * Make the order of Aggregate runs explicit * Sort all instances of crispresso2_folder_info in Aggregate * Sort df_summary_quantification df in Aggregate * Try sorting with a list of single column * Update to correct test branch * Move to master test branch --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_nu…

* Fix CRISPRessoAggregate bug and other improvements (#95) * D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- * Add amplicon_name to plot functions * Add sgRNA sequences to nucleotide quilt parameters in Aggregate * Add custom_colors to Aggregate plot functions * Update Aggregate and make_aggregate_report to have logger and tool * Write command_used to Aggregate .json info file * Point to new test branch and add Aggregate run * Make the order of Aggregate runs explicit * Sort all instances of crispresso2_folder_info in Aggregate * Sort df_summary_quantification df in Aggregate * Try sorting with a list of single column * Update to correct test branch * Move to master test branch --------- * Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns ----… Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]>

* Mckay/c2pro reports test (#99) * Fix CRISPRessoAggregate bug and other improvements (#95) * D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Add amplicon_name to plot functions * Add sgRNA sequences to nucleotide quilt parameters in Aggregate * Add custom_colors to Aggregate plot functions * Update Aggregate and make_aggregate_report to have logger and tool * Write command_used to Aggregate .json info file * Point to new test branch and add Aggregate run * Make the order of Aggregate runs explicit * Sort all instances of crispresso2_folder_info in Aggregate * Sort df_summary_quantification df in Aggregate * Try sorting with a list of single column * Update to correct test branch * Move to master test branch --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_nu…

* Fix CRISPRessoAggregate bug and other improvements (#95) * D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- * Add amplicon_name to plot functions * Add sgRNA sequences to nucleotide quilt parameters in Aggregate * Add custom_colors to Aggregate plot functions * Update Aggregate and make_aggregate_report to have logger and tool * Write command_used to Aggregate .json info file * Point to new test branch and add Aggregate run * Make the order of Aggregate runs explicit * Sort all instances of crispresso2_folder_info in Aggregate * Sort df_summary_quantification df in Aggregate * Try sorting with a list of single column * Update to correct test branch * Move to master test branch --------- * Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns ----… Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]>

* Mckay/c2pro reports test (#99) * Fix CRISPRessoAggregate bug and other improvements (#95) * D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- * Add amplicon_name to plot functions * Add sgRNA sequences to nucleotide quilt parameters in Aggregate * Add custom_colors to Aggregate plot functions * Update Aggregate and make_aggregate_report to have logger and tool * Write command_used to Aggregate .json info file * Point to new test branch and add Aggregate run * Make the order of Aggregate runs explicit * Sort all instances of crispresso2_folder_info in Aggregate * Sort df_summary_quantification df in Aggregate * Try sorting with a list of single column * Update to correct test branch * Move to master test branch --------- * Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_nu… Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]>

* changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]>

* Fix CRISPRessoAggregate bug and other improvements (#95) * D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- * Add amplicon_name to plot functions * Add sgRNA sequences to nucleotide quilt parameters in Aggregate * Add custom_colors to Aggregate plot functions * Update Aggregate and make_aggregate_report to have logger and tool * Write command_used to Aggregate .json info file * Point to new test branch and add Aggregate run * Make the order of Aggregate runs explicit * Sort all instances of crispresso2_folder_info in Aggregate * Sort df_summary_quantification df in Aggregate * Try sorting with a list of single column * Update to correct test branch * Move to master test branch --------- * Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns ----… Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]>

* Mckay/c2pro reports test (#99) * Fix CRISPRessoAggregate bug and other improvements (#95) * D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- * Add amplicon_name to plot functions * Add sgRNA sequences to nucleotide quilt parameters in Aggregate * Add custom_colors to Aggregate plot functions * Update Aggregate and make_aggregate_report to have logger and tool * Write command_used to Aggregate .json info file * Point to new test branch and add Aggregate run * Make the order of Aggregate runs explicit * Sort all instances of crispresso2_folder_info in Aggregate * Sort df_summary_quantification df in Aggregate * Try sorting with a list of single column * Update to correct test branch * Move to master test branch --------- * Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_nu… Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]>

* Fix CRISPRessoAggregate bug and other improvements (#95) * D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- * Add amplicon_name to plot functions * Add sgRNA sequences to nucleotide quilt parameters in Aggregate * Add custom_colors to Aggregate plot functions * Update Aggregate and make_aggregate_report to have logger and tool * Write command_used to Aggregate .json info file * Point to new test branch and add Aggregate run * Make the order of Aggregate runs explicit * Sort all instances of crispresso2_folder_info in Aggregate * Sort df_summary_quantification df in Aggregate * Try sorting with a list of single column * Update to correct test branch * Move to master test branch --------- * Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns ----… Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]>

* Mckay/c2pro reports test (#99) * Fix CRISPRessoAggregate bug and other improvements (#95) * D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- * Add amplicon_name to plot functions * Add sgRNA sequences to nucleotide quilt parameters in Aggregate * Add custom_colors to Aggregate plot functions * Update Aggregate and make_aggregate_report to have logger and tool * Write command_used to Aggregate .json info file * Point to new test branch and add Aggregate run * Make the order of Aggregate runs explicit * Sort all instances of crispresso2_folder_info in Aggregate * Sort df_summary_quantification df in Aggregate * Try sorting with a list of single column * Update to correct test branch * Move to master test branch --------- * Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <[email protected]> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <[email protected]> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <[email protected]> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <[email protected]> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <[email protected]> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <[email protected]> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <[email protected]> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <[email protected]> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <[email protected]> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <[email protected]> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <[email protected]> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <[email protected]> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <[email protected]> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <[email protected]> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <[email protected]> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <[email protected]> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <[email protected]> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <[email protected]> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <[email protected]> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <[email protected]> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <[email protected]> Co-authored-by: mbowcut2 <[email protected]> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <[email protected]> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <[email protected]> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <[email protected]> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <[email protected]> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <[email protected]> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <[email protected]> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <[email protected]> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <[email protected]> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <[email protected]> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <[email protected]> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <[email protected]> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <[email protected]> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <[email protected]> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <[email protected]> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <[email protected]> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <[email protected]> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <[email protected]> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <[email protected]> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <[email protected]> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <[email protected]> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Samuel Nichols <[email protected]> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Cole Lyman <[email protected]> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <[email protected]> Co-authored-by: mbowcut2 <[email protected]> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <[email protected]> --------- Co-authored-by: Cole Lyman <[email protected]> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_nu… Co-authored-by: mbowcut2 <[email protected]> Co-authored-by: Trevor Martin <[email protected]> Co-authored-by: Samuel Nichols <[email protected]>

kclem reviewed Jul 12, 2024

View reviewed changes

CRISPResso2/CRISPRessoPooledCORE.py Outdated Show resolved Hide resolved

kclem requested changes Jul 12, 2024

View reviewed changes

Revert change to pooled mixed mode info statement (#86)

96432ba

kclem approved these changes Jul 15, 2024

View reviewed changes

kclem merged commit 44f692e into pinellolab:master Jul 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement new pooled mixed-mode default behavior #454

Implement new pooled mixed-mode default behavior #454

Colelyman commented Jul 12, 2024

kclem left a comment

Implement new pooled mixed-mode default behavior #454

Implement new pooled mixed-mode default behavior #454

Conversation

Colelyman commented Jul 12, 2024

kclem left a comment

Choose a reason for hiding this comment