Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merging new master commits to develop branch #145

Merged
merged 6 commits into from
Sep 11, 2024
Merged

Merging new master commits to develop branch #145

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
203a855
Added variable for tourmaline path to notebooks, added .gitignore
lukenoaa Aug 14, 2024
70e5034
Fixing .gitignore edits
lukenoaa Aug 14, 2024
2bdbd0c
Merge branch 'aomlomics:master' into master
lukenoaa Aug 14, 2024
a81619c
Merge pull request #143 from lukenoaa/master
lukenoaa Aug 14, 2024
a4ea863
fixed Docker link in README
lukenoaa Sep 2, 2024
66464ed
Merge pull request #144 from lukenoaa/master
lukenoaa Sep 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Ignore Snakemake log files
logs/*.log
Empty file removed 01-imported/.gitignore
View file
Open in desktop
Empty file.
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ To run Tourmaline inside a Docker container:
1. Install Docker Desktop (Mac, Windows, or Linux) from [Docker.com](https://docs.docker.com/get-docker/).
2. Open Docker app.
3. Increase the memory to 8 GB or more (Preferences -> Resources -> Advanced -> Memory).
4. Download the Docker image from [DockerHub](https://hub.docker.com/r/lukenoaa/tourmaline) (command below).
4. Download the Docker image from [DockerHub](https://hub.docker.com/r/aomlomics/tourmaline) (command below).
5. Run the Docker image (command below).

```bash
Expand Down
200 changes: 178 additions & 22 deletions notebooks/tourmaline_meta_analysis.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 1,
"id": "interstate-wrestling",
"metadata": {},
"outputs": [],
Expand All @@ -47,15 +47,33 @@
"## Set file paths"
]
},
{
"cell_type": "markdown",
"id": "eadf279b",
"metadata": {},
"source": [
"**The value of `tourmdir` should be changed if this notebook is not being run from its original location in the `notebooks` directory.**"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 7,
"id": "3f259b06",
"metadata": {},
"outputs": [],
"source": [
"tourmdir = '..'"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "sustainable-round",
"metadata": {},
"outputs": [],
"source": [
"dir_analysis1 = '../meta-analysis/analysis1'\n",
"dir_analysis2 = '../meta-analysis/analysis2'"
"dir_analysis1 = '%s/meta-analysis/analysis1' % tourmdir\n",
"dir_analysis2 = '%s/meta-analysis/analysis2' % tourmdir"
]
},
{
Expand All @@ -76,34 +94,172 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 3,
"id": "fancy-soccer",
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Cloning into 'tourmaline'...\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Removed:\n",
"00-data/fastq/Sample_134723_R2.fastq.gz\n",
"00-data/fastq/Sample_134722_R1.fastq.gz\n",
"00-data/fastq/Sample_134789_R2.fastq.gz\n",
"00-data/fastq/Sample_134719_R1.fastq.gz\n",
"00-data/fastq/Sample_134768_R2.fastq.gz\n",
"00-data/fastq/Sample_134718_R2.fastq.gz\n",
"00-data/fastq/Sample_134771_R2.fastq.gz\n",
"00-data/fastq/Sample_134790_R2.fastq.gz\n",
"00-data/fastq/Sample_134778_R2.fastq.gz\n",
"00-data/fastq/Sample_134767_R2.fastq.gz\n",
"00-data/fastq/Sample_134717_R2.fastq.gz\n",
"00-data/fastq/Sample_134716_R1.fastq.gz\n",
"00-data/fastq/Sample_134772_R2.fastq.gz\n",
"00-data/fastq/Sample_134777_R2.fastq.gz\n",
"00-data/fastq/Sample_134721_R1.fastq.gz\n",
"00-data/fastq/Sample_134720_R2.fastq.gz\n",
"00-data/fastq/Sample_134778_R1.fastq.gz\n",
"00-data/fastq/Sample_134771_R1.fastq.gz\n",
"00-data/fastq/Sample_134790_R1.fastq.gz\n",
"00-data/fastq/Sample_134789_R1.fastq.gz\n",
"00-data/fastq/Sample_134719_R2.fastq.gz\n",
"00-data/fastq/Sample_134768_R1.fastq.gz\n",
"00-data/fastq/Sample_134718_R1.fastq.gz\n",
"00-data/fastq/Sample_134723_R1.fastq.gz\n",
"00-data/fastq/Sample_134722_R2.fastq.gz\n",
"00-data/fastq/Sample_134721_R2.fastq.gz\n",
"00-data/fastq/Sample_134720_R1.fastq.gz\n",
"00-data/fastq/Sample_134777_R1.fastq.gz\n",
"00-data/fastq/Sample_134772_R1.fastq.gz\n",
"00-data/fastq/Sample_134767_R1.fastq.gz\n",
"00-data/fastq/Sample_134717_R1.fastq.gz\n",
"00-data/fastq/Sample_134716_R2.fastq.gz\n",
"00-data/fastq\n",
"00-data/manifest_pe.csv\n",
"00-data/manifest_se.csv\n",
"00-data/metadata.tsv\n",
"00-data/repseqs_to_filter_dada2-pe.tsv\n",
"00-data/repseqs_to_filter_dada2-se.tsv\n",
"00-data/repseqs_to_filter_deblur-se.tsv\n",
"00-data/samples_to_filter_dada2-pe.tsv\n",
"00-data/samples_to_filter_dada2-se.tsv\n",
"00-data/samples_to_filter_deblur-se.tsv\n",
"Copied:\n",
"../../../config.yaml -> config.yaml\n",
"../../../00-data/manifest_pe.csv -> 00-data/manifest_pe.csv\n",
"../../../00-data/manifest_se.csv -> 00-data/manifest_se.csv\n",
"../../../00-data/metadata.tsv -> 00-data/metadata.tsv\n",
"../../../00-data/repseqs_to_filter_dada2-pe.tsv -> 00-data/repseqs_to_filter_dada2-pe.tsv\n",
"../../../00-data/repseqs_to_filter_dada2-se.tsv -> 00-data/repseqs_to_filter_dada2-se.tsv\n",
"../../../00-data/repseqs_to_filter_deblur-se.tsv -> 00-data/repseqs_to_filter_deblur-se.tsv\n",
"../../../01-imported/refseqs.qza -> 01-imported/refseqs.qza\n",
"../../../01-imported/reftax.qza -> 01-imported/reftax.qza\n"
]
}
],
"source": [
"%%bash\n",
"cd ../meta-analysis/analysis1\n",
"git clone https://github.com/aomlomics/tourmaline\n",
"cd tourmaline\n",
"./scripts/initialize_dir_from_existing_tourmaline_dir.sh ../../..\n",
"cp ../metadata.tsv 00-data\n",
"cp ../manifest_* 00-data"
"cp ../manifest_* 00-data\n",
"cp ../../../01-imported/silva-138-99-seqs-515-806.qza 01-imported\n",
"cp ../../../01-imported/silva-138-99-tax-515-806.qza 01-imported"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 4,
"id": "manual-linux",
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Cloning into 'tourmaline'...\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Removed:\n",
"00-data/fastq/Sample_134723_R2.fastq.gz\n",
"00-data/fastq/Sample_134722_R1.fastq.gz\n",
"00-data/fastq/Sample_134789_R2.fastq.gz\n",
"00-data/fastq/Sample_134719_R1.fastq.gz\n",
"00-data/fastq/Sample_134768_R2.fastq.gz\n",
"00-data/fastq/Sample_134718_R2.fastq.gz\n",
"00-data/fastq/Sample_134771_R2.fastq.gz\n",
"00-data/fastq/Sample_134790_R2.fastq.gz\n",
"00-data/fastq/Sample_134778_R2.fastq.gz\n",
"00-data/fastq/Sample_134767_R2.fastq.gz\n",
"00-data/fastq/Sample_134717_R2.fastq.gz\n",
"00-data/fastq/Sample_134716_R1.fastq.gz\n",
"00-data/fastq/Sample_134772_R2.fastq.gz\n",
"00-data/fastq/Sample_134777_R2.fastq.gz\n",
"00-data/fastq/Sample_134721_R1.fastq.gz\n",
"00-data/fastq/Sample_134720_R2.fastq.gz\n",
"00-data/fastq/Sample_134778_R1.fastq.gz\n",
"00-data/fastq/Sample_134771_R1.fastq.gz\n",
"00-data/fastq/Sample_134790_R1.fastq.gz\n",
"00-data/fastq/Sample_134789_R1.fastq.gz\n",
"00-data/fastq/Sample_134719_R2.fastq.gz\n",
"00-data/fastq/Sample_134768_R1.fastq.gz\n",
"00-data/fastq/Sample_134718_R1.fastq.gz\n",
"00-data/fastq/Sample_134723_R1.fastq.gz\n",
"00-data/fastq/Sample_134722_R2.fastq.gz\n",
"00-data/fastq/Sample_134721_R2.fastq.gz\n",
"00-data/fastq/Sample_134720_R1.fastq.gz\n",
"00-data/fastq/Sample_134777_R1.fastq.gz\n",
"00-data/fastq/Sample_134772_R1.fastq.gz\n",
"00-data/fastq/Sample_134767_R1.fastq.gz\n",
"00-data/fastq/Sample_134717_R1.fastq.gz\n",
"00-data/fastq/Sample_134716_R2.fastq.gz\n",
"00-data/fastq\n",
"00-data/manifest_pe.csv\n",
"00-data/manifest_se.csv\n",
"00-data/metadata.tsv\n",
"00-data/repseqs_to_filter_dada2-pe.tsv\n",
"00-data/repseqs_to_filter_dada2-se.tsv\n",
"00-data/repseqs_to_filter_deblur-se.tsv\n",
"00-data/samples_to_filter_dada2-pe.tsv\n",
"00-data/samples_to_filter_dada2-se.tsv\n",
"00-data/samples_to_filter_deblur-se.tsv\n",
"Copied:\n",
"../../../config.yaml -> config.yaml\n",
"../../../00-data/manifest_pe.csv -> 00-data/manifest_pe.csv\n",
"../../../00-data/manifest_se.csv -> 00-data/manifest_se.csv\n",
"../../../00-data/metadata.tsv -> 00-data/metadata.tsv\n",
"../../../00-data/repseqs_to_filter_dada2-pe.tsv -> 00-data/repseqs_to_filter_dada2-pe.tsv\n",
"../../../00-data/repseqs_to_filter_dada2-se.tsv -> 00-data/repseqs_to_filter_dada2-se.tsv\n",
"../../../00-data/repseqs_to_filter_deblur-se.tsv -> 00-data/repseqs_to_filter_deblur-se.tsv\n",
"../../../01-imported/refseqs.qza -> 01-imported/refseqs.qza\n",
"../../../01-imported/reftax.qza -> 01-imported/reftax.qza\n"
]
}
],
"source": [
"%%bash\n",
"cd ../meta-analysis/analysis2\n",
"git clone https://github.com/aomlomics/tourmaline\n",
"cd tourmaline\n",
"./scripts/initialize_dir_from_existing_tourmaline_dir.sh ../../..\n",
"cp ../metadata.tsv 00-data\n",
"cp ../manifest_* 00-data"
"cp ../manifest_* 00-data\n",
"cp ../../../01-imported/silva-138-99-seqs-515-806.qza 01-imported\n",
"cp ../../../01-imported/silva-138-99-tax-515-806.qza 01-imported"
]
},
{
Expand All @@ -127,33 +283,33 @@
"id": "helpful-snapshot",
"metadata": {},
"source": [
"Each commands below will take ~5 minutes to complete. You will not see any output until the command is finished."
"Run the two commands below from your Terminal (i.e., outside of this notebook). Each command will take ~5 minutes to complete."
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"id": "cathedral-professor",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"%%bash\n",
"```bash\n",
"conda activate snakemake\n",
"cd ../meta-analysis/analysis1/tourmaline\n",
"snakemake dada2_pe_report_unfiltered"
"snakemake --use-conda dada2_pe_report_unfiltered --cores 4\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"id": "brazilian-skating",
"metadata": {},
"outputs": [],
"source": [
"%%bash\n",
"```bash\n",
"conda activate snakemake\n",
"cd ../meta-analysis/analysis2/tourmaline\n",
"snakemake dada2_pe_report_unfiltered"
"snakemake --use-conda dada2_pe_report_unfiltered --cores 4\n",
"```"
]
},
{
Expand Down Expand Up @@ -365,7 +521,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.13"
"version": "3.8.16"
}
},
"nbformat": 4,
Expand Down
78 changes: 45 additions & 33 deletions notebooks/tourmaline_python_notebook.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

Loading