This paper review was conducted as part of the seminar "Large Language Models."
Language models are increasingly being used to solve complex reasoning tasks. However, they often struggle with these tasks because they can generate several different answers that are consistent with the prompt. This can make it difficult for the language model to select the correct answer. In this paper, I provide an analysis of a new decoding strategy called self-consistency. This strategy tackles a problem by sampling various reasoning paths and selecting the most consistent answer. The paper evaluates self-consistency decoding on a number of arithmetic and common sense reasoning benchmarks and shows that it significantly improves the performance of language models on these tasks. In addition, the limitations of the self-consistency method are discussed, and future research directions are suggested.
The original paper can be found here: Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2023). Self-Consistency Improves Chain of Thought Reasoning in Language Models. arXiv preprint arXiv:2203.11171.