Implementation of the paper 'GAPX: Generalized Autoregressive Paraphrase-identification X'
NeurIPS 2022
An ensemble model for paraphrase identification robust to distribution shift.
- GPU
- requirements.txt
Please download the following paraphrase identification datasets:
- Quora Question Pair : https://quoradata.quora.com/First-Quora-Dataset-Release-Question-Pairs
- Paraphrase and Semantic Similarity on Twitter: https://github.com/cocoxu/SemEval-PIT2015
- Paraphrase Adversarials from Word Scrambling (en): https://www.statmt.org/wmt17/metrics-task.html
- WMT 17: https://www.statmt.org/wmt17/metrics-task.html
To train and evaluate a paraphrase identification model, run:
python run.py --source_dataset [QQP, PIT, PAWS] --option [naive, robust]
Here we implemented a simplified version from the paper, where for the discriminative model, we use BART instead of RoBERTa
You should expect to see something similar to this (f1/acc/auc):
| Command | QQP->QQP | QQP->WMT | QQP->PAWS | QQP->PIT |
|---|---|---|---|---|
python run.py --source_dataset QQP --option naive |
83.4/83.5/91.2 | 66.7/66.8/74.2 | 44.7/49.8/57.1 | 63.6/66.5/82.0 |
python run.py --source_dataset QQP --option robust |
83.1/83.2/88.4 | 74.4/74.7/79.3 | 56.6/56.9/59.5 | 62.3/63.6/73.5 |