Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does Decagon regard the relation and its corresponding reverse one as different relations? #28

Open
ZillaRU opened this issue Jan 5, 2022 · 2 comments

Comments

@ZillaRU
Copy link

ZillaRU commented Jan 5, 2022

Typically, DDI prediction is a pairwise classification problem.
In the given toy example, you artificially generate a small graph.
My concern is that Decagon seems to treat a DDI and its corresponding reverse one as two different edges. But, I am confused by how to calculate the metrics with the predicted results for potential DDI triplets and the reverse ones.
For instance, "Drug A's metabolism is increased when combined with Drug B" (symbolized as triplet (A, metabolism increased, B)) is semantically equal to the reverse one "Drug B can increase the metabolism of Drug A" (symbolized as triplet (B, increase metabolism, A)).

decagon/main.py

Lines 140 to 145 in 86ff6b1

adj_mats_orig = {
(0, 0): [gene_adj, gene_adj.transpose(copy=True)],
(0, 1): [gene_drug_adj],
(1, 0): [drug_gene_adj],
(1, 1): drug_drug_adj_list + [x.transpose(copy=True) for x in drug_drug_adj_list],
}

If the model gives different scores for both triplets, how to calculate the final metric values. By simply keeping predicted results for both groups?

@ZillaRU
Copy link
Author

ZillaRU commented Jan 9, 2022

I have run this code on my real-world datasets and got the predicted result.
Since Decagon regards the relation and its corresponding reverse one as different relations, for each triplet, there are two predicted scores. Did you calculate the performance metric values reported in your paper based on both?

@ZillaRU
Copy link
Author

ZillaRU commented Jan 9, 2022

I have run this code on my real-world datasets and got the predicted result. Since Decagon regards the relation and its corresponding reverse one as different relations, for each triplet, there are two predicted scores. Did you calculate the performance metric values reported in your paper based on both?

decagon/main.py

Lines 295 to 302 in 86ff6b1

for et in range(num_edge_types):
roc_score, auprc_score, apk_score = get_accuracy_scores(
minibatch.test_edges, minibatch.test_edges_false, minibatch.idx2edge_type[et])
print("Edge type=", "[%02d, %02d, %02d]" % minibatch.idx2edge_type[et])
print("Edge type:", "%04d" % et, "Test AUROC score", "{:.5f}".format(roc_score))
print("Edge type:", "%04d" % et, "Test AUPRC score", "{:.5f}".format(auprc_score))
print("Edge type:", "%04d" % et, "Test AP@k score", "{:.5f}".format(apk_score))
print()

But, you seem to calculate the metrics for the relation and the corresponding reverse one separately. Finally, which is reported in your paper?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant