Recall/Precision by SVType

[ACEnglish]

truvari bench will report overall statistics in the output directory's 'summary.txt' file. Often, we want to look at subsets of the data to better understand how our tools are performing. As an example, here we'll show how to report the recall and precision by SVTYPE.

Run vcf2df on a bench output directory

truvari vcf2df --bench-dir --format --info bench_dir/ data.jl

Then, run a jupyter notebook or make a script.

# Setup Imports
import joblib
import pandas as pd
import seaborn as sb
sb.set()

data = joblib.load("data.jl")

# Plot SV counts by svtype and state
p = sb.countplot(data=data, hue="svtype", x="state", hue_order=["DEL", "INS"])
hide = p.set(title="SVCount by State")

# Make a table
rows = []
for label, counts in data.groupby(["svtype"]):
    counts = counts["state"].value_counts()
    precision = counts["tp"] / (counts["tp"] + counts["fp"])
    recall = counts["tpbase"] / (counts["tpbase"] + counts["fn"])
    rows.append([label, recall, precision])

counts = data["state"].value_counts()
precision = counts["tp"] / (counts["tp"] + counts["fp"])
recall = counts["tpbase"] / (counts["tpbase"] + counts["fn"])
rows.append(["Total", recall, precision])

result = pd.DataFrame(rows, columns=["svtype", "recall", "precision"])
# Dropping NA so that truvari.SV types that don't have calls aren't seen
print(result.dropna())

Results: