PathQuestion[1] includes two subsets, PQ and PQL, constructed by adopting two subsets of Freebase as Knowledge Bases. In this dataset, paths are extracted between two entities which span two hops (denoted by -2H) or three hops (denoted by -3H) and then generated natural language questions with templates. To make the generated questions analogical to real-world questions, paraphrasing templates and synonyms for relations are included by searching the Internet and two real-world datasets, WebQuestions[2] and WikiAnswers[3]. In this way, the syntactic structure and surface wording of the generated questions have been greatly enriched.
Please see the original paper for more details about the dataset.
This dataset can be downloaded via the link.
| Model / System | Year | Precision | Recall | F1 | Language | Reported by |
|---|---|---|---|---|---|---|
| ISM | 2022 | - | - | 99.1(Hits@1) | EN | AlAgha, 2022 |
| QAGCN | 2022 | - | - | 98.5(Hits@1) | EN | Wang et al. |
| Uhop-HR | 2022 | - | - | 97.6(Hits@1) | EN | AlAgha, 2022 |
| AlAgha, 2022 | 2022 | - | - | 97.4(Hits@1) | EN | AlAgha, 2022 |
| SRN | 2022 | - | - | 96.3(Hits@1) | EN | Wang et al. |
| RL-MHR | 2022 | - | - | 94.1(Hits@1) | EN | AlAgha, 2022 |
| TransferNet | 2022 | - | - | 93.2(Hits@1) | EN | AlAgha, 2022 |
| IRN | 2022 | - | - | 91.9(Hits@1) | EN | Wang et al. |
| IRN | 2022 | - | - | 78.3(Hits@1) | EN | AlAgha, 2022 |
| HR-BiLSTM | 2022 | - | - | 76.8(Hits@1) | EN | AlAgha, 2022 |
| MINERVA | 2022 | - | - | 75.9(Hits@1) | EN | Wang et al. |
| Model / System | Year | Precision | Recall | F1 | Accuracy | Language | Reported by |
|---|---|---|---|---|---|---|---|
| AlAgha, 2022 | 2022 | - | - | 92.3(Hits@1) | - | EN | AlAgha, 2022 |
| Edge-aware GNN | 2022 | - | - | 85.6(Hits@1) | - | EN | Zhang |
| ISM | 2022 | - | - | 84.9(Hits@1) | - | EN | AlAgha, 2022 |
| TransferNet | 2022 | - | - | 84.1(Hits@1) | - | EN | AlAgha, 2022 |
| Uhop-HR | 2022 | - | - | 82.6(Hits@1) | - | EN | AlAgha, 2022 |
| RL-MHR | 2022 | - | - | 82.2(Hits@1) | - | EN | AlAgha, 2022 |
| GlobalGraph | 2022 | - | - | 76.0(Hits@1) | - | EN | Zhang |
| 2HR-DR | 2022 | - | - | 75.5(Hits@1) | - | EN | Zhang |
| IRN | 2022 | - | - | 72.5(Hits@1) | - | EN | Zhang |
| SGReader | 2022 | - | - | 71.9(Hits@1) | - | EN | Zhang |
| HR-BiLSTM | 2022 | - | - | 71.9(Hits@1) | - | EN | AlAgha, 2022 |
| GRAFT-Net | 2022 | - | - | 70.7(Hits@1) | - | EN | Zhang |
| IRN | 2022 | - | - | 66.2(Hits@1) | - | EN | AlAgha, 2022 |
| KV-MemNN | 2022 | - | - | 62.2(Hits@1) | - | EN | Zhang |
| MRP-QA-marginal_prob | 2022 | - | - | - | 98.4 | EN | Wang et al. |
| UHop | 2022 | - | - | - | 97.5 | EN | Wang et al. |
| HR-BiLSTM | 2022 | - | - | - | 97.5 | EN | Wang et al. |
| ABWIM | 2022 | - | - | - | 94.3 | EN | Wang et al. |
| KV-MemNN | 2022 | - | - | - | 72.2 | EN | Wang et al. |
| Model / System | Year | Precision | Recall | F1 | Language | Reported by |
|---|---|---|---|---|---|---|
| AlAgha, 2022 | 2022 | - | - | 98.7(Hits@1) | EN | AlAgha, 2022 |
| ISM | 2022 | - | - | 95.7(Hits@1) | EN | AlAgha, 2022 |
| TransferNet | 2022 | - | - | 91.3(Hits@1) | EN | AlAgha, 2022 |
| Uhop-HR | 2022 | - | - | 91.3(Hits@1) | EN | AlAgha, 2022 |
| QAGCN | 2022 | - | - | 90.6(Hits@1) | EN | Wang et al. |
| SRN | 2022 | - | - | 89.2(Hits@1) | EN | Wang et al. |
| RL-MHR | 2022 | - | - | 87.2(Hits@1) | EN | AlAgha, 2022 |
| IRN | 2022 | - | - | 83.3(Hits@1) | EN | Wang et al. |
| IRN | 2022 | - | - | 74.3(Hits@1) | EN | AlAgha, 2022 |
| HR-BiLSTM | 2022 | - | - | 74.1(Hits@1) | EN | AlAgha, 2022 |
| MINERVA | 2022 | - | - | 71.2(Hits@1) | EN | Wang et al. |
| Model / System | Year | Precision | Recall | F1 | Accuracy | Language | Reported by |
|---|---|---|---|---|---|---|---|
| GlobalGraph | 2022 | - | - | 94.1(Hits@1) | - | EN | Zhang |
| Edge-aware GNN | 2022 | - | - | 93.1(Hits@1) | - | EN | Zhang |
| 2HR-DR | 2022 | - | - | 92.1(Hits@1) | - | EN | Zhang |
| GRAFT-Net | 2022 | - | - | 91.3(Hits@1) | - | EN | Zhang |
| AlAgha, 2022 | 2022 | - | - | 89.7(Hits@1) | - | EN | AlAgha, 2022 |
| SGReader | 2022 | - | - | 89.3(Hits@1) | - | EN | Zhang |
| TransferNet | 2022 | - | - | 82.7(Hits@1) | - | EN | AlAgha, 2022 |
| ISM | 2022 | - | - | 81.7(Hits@1) | - | EN | AlAgha, 2022 |
| Uhop-HR | 2022 | - | - | 80.1(Hits@1) | - | EN | AlAgha, 2022 |
| RL-MHR | 2022 | - | - | 77.8(Hits@1) | - | EN | AlAgha, 2022 |
| IRN | 2022 | - | - | 71.0(Hits@1) | - | EN | Zhang |
| KV-MemNN | 2022 | - | - | 67.4(Hits@1) | - | EN | Zhang |
| HR-BiLSTM | 2022 | - | - | 61.6(Hits@1) | - | EN | AlAgha, 2022 |
| IRN | 2022 | - | - | 59.1(Hits@1) | - | EN | AlAgha, 2022 |
| MRP-QA-marginal_prob | 2022 | - | - | - | 97.8 | EN | Wang et al. |
| UHop | 2022 | - | - | - | 89.3 | EN | Wang et al. |
| ABWIM | 2022 | - | - | - | 89.3 | EN | Wang et al. |
| HR-BiLSTM | 2022 | - | - | - | 87.9 | EN | Wang et al. |
[1] Zhou, Mantong, Minlie Huang, and Xiaoyan Zhu. An interpretable reasoning network for multi-relation question answering. arXiv preprint arXiv:1801.04726 (2018).
[2] Berant, Jonathan, Andrew Chou, Roy Frostig, and Percy Liang. Semantic parsing on freebase from question-answer pairs. In Proceedings of the 2013 conference on empirical methods in natural language processing, pp. 1533-1544. 2013.
[3] Fader, Anthony, Luke Zettlemoyer, and Oren Etzioni. Paraphrase-driven learning for open question answering. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1608-1618. 2013.