Skip to content

Latest commit

 

History

History
92 lines (74 loc) · 4.05 KB

README.md

File metadata and controls

92 lines (74 loc) · 4.05 KB
Raw

Discovering New Intents with Deep Aligned Clustering

A deep aligned clustering method to discover new intents.

The proposed method together with baselines are also integrated into the open intent discovery module in our another scalable framework TEXTOIR, enjoy it!

Introduction

This repository provides the official PyTorch implementation of the research paper Discovering New Intents with Deep Aligned Clustering (Accepted by AAAI2021).

Related works can refer to the reading list.

Dependencies

We use anaconda to create python environment:

conda create --name python=3.6

Install all required libraries:

pip install -r requirements.txt

Model Preparation

Get the pre-trained BERT model and convert it into Pytorch.

Set the path of the uncased-bert model (parameter "bert_model" in init_parameter.py).

Usage

Run the experiments by:

sh scripts/run.sh

You can change the parameters in the script. The selected parameters are as follows:

dataset: clinc | banking
factor_of_clusters: 1 (default) | 2 | 3 | 4 
known_class_ratio: 0.25 | 0.5 | 0.75 (default)

Model

The model architecture of DeepAligned: Model

Results

The detailed results can be seen in known_intent_ratio_results.csv and k_results.csv.

Main experiments

CLINC BANKING
Method NMI ARI ACC NMI ARI ACC
KM 70.89 26.86 45.06 54.57 12.18 29.55
AG 73.07 27.70 44.03 57.07 13.31 31.58
SAE-KM 73.13 29.95 46.75 63.79 22.85 38.92
DEC 74.83 27.46 46.89 67.78 27.21 41.29
DCN 75.66 31.15 49.29 67.54 26.81 41.99
DAC 78.40 40.49 55.94 47.35 14.24 27.41
DeepCluster 65.58 19.11 35.70 41.77 8.95 20.69
PCK-means 68.70 35.40 54.61 48.22 16.24 32.66
BERT-KCL 86.82 58.79 68.86 75.21 46.72 60.15
BERT-MCL 87.72 59.92 69.66 75.68 47.43 61.14
CDAC+ 86.65 54.33 69.89 72.25 40.97 53.83
BERT-DTC 90.54 65.02 74.15 76.55 44.70 56.51
DeepAligned 93.89 79.75 86.49 79.56 53.64 64.90

Ablation study

Method CLINC BANKING
Method NMI ARI ACC NMI ARI ACC
w/o Pre + Reinit 57.80 9.63 23.02 34.34 4.49 13.67
w/o Pre + Align 62.53 14.10 28.63 36.91 5.23 15.42
Pre + Reinit 82.90 45.67 55.80 68.12 31.56 41.32
Pre + Align 93.89 79.75 86.49 79.56 53.64 64.90

If you are insterested in this work, and want to use the codes or results in this repository, please star this repository and cite by:

@article{Zhang_Xu_Lin_Lyu_2021, 
    title={Discovering New Intents with Deep Aligned Clustering}, 
    volume={35}, 
    number={16}, 
    journal={Proceedings of the AAAI Conference on Artificial Intelligence}, 
    author={Zhang, Hanlei and Xu, Hua and Lin, Ting-En and Lyu, Rui}, 
    year={2021}, 
    month={May}, 
    pages={14365-14373}
}

Acknowledgments

This paper is founded by seed fund of Tsinghua University (Department of Computer Science and Technology)- Siemens Ltd., China Joint Research Center for Industrial Intelligence and Internet of Things.