WebQuestionsSP[1] is the annotated version of WebQuestions, which contains questions that require a 1 or 2-hop relation path to arrive at the answer entity. More specifically, about 40% of the questions require a 2-hop relation to reach the answer. Freebase is its backgound KB. Originally, it splits into 3,298 questions as train set and 1,639 questions as test set.
This dataset can be downloaded via the link.
| Model / System | Year | F1 | Hits@1 | Accuracy | Language | Reported by |
|---|---|---|---|---|---|---|
| chatGPT | 2023 | - | - | 83.70 | EN | Tan et al. |
| TIARA | 2022 | 78.9 | 75.2 | - | EN | Shu et. al. |
| DECAF (DPR + FiD-3B) | 2022 | 78.8 | 82.1 | - | EN | Yu et al. |
| GPT-3.5v3 | 2023 | - | - | 79.60 | EN | Tan et al. |
| DECAF (DPR + FiD-large) | 2022 | 77.1 ± 0.2 | 80.7 ± 0.2 | - | EN | Yu et al. |
| UniK-QA | 2022 | - | 79.1 | - | EN | Yu et al. |
| SQALER+GNN | 2022 | - | 76.1 | - | EN | Costas Mavromatis and George Karypis |
| EmQL | 2020 | - | 75.5 | - | EN | Yu et al. |
| GMT-KBQA | 2022 | 76.6 | - | 73.1 | EN | Hu et al. |
| GPT-3.5v2 | 2023 | - | - | 72.34 | EN | Tan et al. |
| Program Transfer | 2022 | 76.5 | 74.6 | - | EN | Yu et al. |
| RnG-KBQA (T5-large) | 2022 | 76.2 ± 0.2 | 80.7 ± 0.2 | - | EN | Yu et al. |
| RnG-KBQA | 2022 | 75.6 | - | 71.1 | EN | Hu et al. |
| ArcaneQA | 2022 | 75.3 | - | - | EN | Yu et al. |
| QNRKGQA+h | 2022 | - | 75.7 | - | EN | Ma et al. |
| DECAF (BM25 + FiD-large) | 2022 | 74.9 ± 0.3 | 79.0 ± 0.4 | - | EN | Yu et al. |
| MRP-QA-marginal_prob | 2022 | 74.9 | - | - | EN | Wang et al. |
| QNRKGQA | 2022 | - | 74.9 | - | EN | Ma et al. |
| ReTrack | 2022 | 74.7 | - | - | EN | Hu et al. |
| ReTrack | 2021 | 74.6 | 74.7 | - | EN | Yu et al. |
| BART-large | 2022 | 74.6 | - | - | EN | Hu et al. |
| Subgraph Retrieval | 2022 | 74.5 | 83.2 | - | EN | Shu et. al. |
| QGG | 2022 | 74.0 | - | - | EN | Yu et al. |
| CBR-KBQA | 2021 | 72.8 | - | 69.9 | EN | Yu et al. |
| GPT-3 | 2023 | - | - | 67.78 | EN | Tan et al. |
| KGQA-RR(Roberta) | 2023 | - | - | 64.59 | EN | Hu et al. |
| KGQA-RR(Luke) | 2023 | - | - | 64.52 | EN | Hu et al. |
| KGQA-RR(Kepler) | 2023 | - | - | 64.46 | EN | Hu et al. |
| KGQA-RR(Bert) | 2023 | - | - | 64.11 | EN | Hu et al. |
| KGQA-RR(Albert) | 2023 | - | - | 63.89 | EN | Hu et al. |
| KGQA-RR(XLnet) | 2023 | - | - | 63.87 | EN | Hu et al. |
| KGQA-RR(DistilBert) | 2023 | - | - | 63.59 | EN | Hu et al. |
| KGQA-RR(DistilRoberta) | 2023 | - | - | 62.57 | EN | Hu et al. |
| KGQA-CL(Roberta) | 2023 | - | - | 62.32 | EN | Hu et al. |
| KGQA-CL(Luke) | 2023 | - | - | 62.31 | EN | Hu et al. |
| KGQA-CL(Kepler) | 2023 | - | - | 62.02 | EN | Hu et al. |
| KGQA-CL(Bert) | 2023 | - | - | 61.76 | EN | Hu et al. |
| KGQA-CL(DistilBert) | 2023 | - | - | 61.49 | EN | Hu et al. |
| KGQA-CL(Albert) | 2023 | - | - | 61.47 | EN | Hu et al. |
| KGQA-CL(XLnet) | 2023 | - | - | 61.46 | EN | Hu et al. |
| KGQA-CL(DistilRoberta) | 2023 | - | - | 61.05 | EN | Hu et al. |
| KGQA-CL(GPT2) | 2023 | - | - | 60.49 | EN | Hu et al. |
| NSM | 2021 | - | 74.30 | - | EN | He et al. |
| Rigel | 2022 | - | 73.3 | - | EN | Costas Mavromatis and George Karypis |
| SGM | 2022 | 72.36 | - | - | EN | Ma L et al. |
| CBR-SUBG | 2022 | 72.1 | - | - | EN | Yu et al. |
| NPI | 2022 | - | 72.6 | - | EN | Cao et al. |
| TextRay | 2022 | - | 72.2 | - | EN | Cao et al. |
| CBR-SUBG | 2022 | - | 72.10 | - | EN | Das et al. |
| KGQA Based on Query Path Generation | 2022 | - | 71.7 | - | EN | Yang et al. |
| STAGG_SP | 2022 | 71.7 | - | - | EN | Wang et al. |
| SSKGQA | 2022 | - | 71.4 | - | EN | Mingchen Li and Jonathan Shihao Ji |
| TransferNet | 2022 | - | 71.4 | - | EN | Shi et al. |
| SeqM | 2020 | 71.83 | - | - | EN | Ma L et al. |
| ReTraCK | 2021 | 71.0 | 71.6 | - | EN | Shu et. al. |
| REAREV | 2022 | 70.9 | 76.4 | - | EN | Costas Mavromatis and George Karypis |
| HGNet | 2021 | 70.3 | 70.6 | - | EN | Yu et al. |
| GrailQA Ranking | 2021 | 70.0 | - | - | EN | Shu et. al. |
| SQALER | 2022 | - | 70.6 | - | EN | Costas Mavromatis and George Karypis |
| STAGG | 2015 | 69.00 | - | - | EN | Ma L et al. |
| UHop | 2019 | 68.5 | - | - | EN | Ma L et al. |
| KBIGER | 2022 | 68.4 | 75.3 | - | EN | Du et al. |
| NSM | 2022 | - | 69.0 | - | EN | Cao et al. |
| GraftNet-EF+LF | 2018 | - | 68.7 | - | EN | Sun et al. |
| PullNet | 2019 | - | 68.1 | - | EN | Sun et al. |
| KBQA-GST | 2022 | 67.9 | - | - | EN | Wang et al. |
| Topic Units | 2019 | 67.9 | - | - | EN | Ma L et al. |
| NSM | 2022 | 67.4 | 74.3 | - | EN | Du et al. |
| Relation Learning | 2021 | 64.5 | 72.9 | - | EN | Shu et. al. |
| SR+NSM | 2022 | 64.1 | 69.5 | - | EN | Yu et al. |
| NSM | 2022 | 62.8 | 68.7 | - | EN | Costas Mavromatis and George Karypis |
| ARN_ConvE | 2023 | - | 68.0 | - | EN | Cui et al. |
| GraftNet | 2022 | 62.8 | 67.8 | - | EN | Du et al. |
| PullNet | 2019 | 62.8 | 67.8 | - | EN | Yu et al. |
| DCRN | 2021 | - | 67.8 | - | EN | Cai et al. |
| ARN_TuckER | 2023 | - | 67.5 | - | EN | Cui et al. |
| NRQA | 2022 | - | 67.1 | - | EN | Guo et al. |
| GraftNet | 2022 | - | 66.4 | - | EN | Mingchen Li and Jonathan Shihao Ji |
| EmbedKGQA | 2020 | - | 66.6 | - | EN | Saxena et al. |
| GraftNet | 2022 | 62.4 | 66.7 | - | EN | Costas Mavromatis and George Karypis |
| HR-BiLSTM | 2022 | 62.3 | - | - | EN | Wang et al. |
| GraftNet-EF+LF | 2018 | 62.30 | - | - | EN | Sun et al. |
| TextRay | 2019 | 60.3 | - | - | EN | Bhutani et al. |
| SGReader | 2022 | 57.3 | 67.2 | - | EN | Costas Mavromatis and George Karypis |
| ARN_ComplEx | 2023 | - | 65.3 | - | EN | Cui et al. |
| ARN_DistMult | 2023 | - | 61.7 | - | EN | Cui et al. |
| FLAN-T5 | 2023 | - | - | 59.87 | EN | Tan et al. |
| KGT5 | 2022 | 56.1 | - | - | EN | Yu et al. |
| FILM | 2022 | 54.7 | - | - | EN | Yu et al. |
| ReifKB | 2020 | - | 52.7 | - | EN | Cohen et al. |
| KV-Mem | 2022 | 38.6 | 46.7 | - | EN | Du et al. |
| KGQA-RR(GPT2) | 2023 | - | - | 18.11 | EN | Hu et al. |
[1] Yih, Wen-tau, Matthew Richardson, Christopher Meek, Ming-Wei Chang, and Jina Suh. The value of semantic parse labeling for knowledge base question answering. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 201-206. 2016.