🔗 Puzzle Solving using Reasoning of Large Language Models: A Survey

📝 Citation

@misc{giadikiaroglou2024puzzle,
      title={Puzzle Solving using Reasoning of Large Language Models: A Survey}, 
      author={Panagiotis Giadikiaroglou and Maria Lymperaiou and Giorgos Filandrianos and Giorgos Stamou},
      year={2024},
      eprint={2402.11291},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Abstract

Exploring the capabilities of Large Language Models (LLMs) in puzzle solving unveils critical insights into their potential and challenges in artificial intelligence, marking a significant step towards understanding their applicability in complex reasoning tasks. This survey leverages a unique taxonomy – dividing puzzles into rule-based and rule-less categories – to critically assess LLMs through various methodologies, including prompting techniques, neuro-symbolic approaches, and fine-tuning. Through a critical review of relevant datasets and benchmarks, we assess LLMs’ performance, identifying significant challenges in complex puzzle scenarios. Our findings highlight the disparity between LLM capabilities and human-like reasoning, particularly in those requiring advanced logical inference. The survey underscores the necessity for novel strategies and richer datasets to advance LLMs’ puzzle-solving proficiency and contribute to AI’s logical reasoning and creative problem-solving advancements.

Laboratory

Artificial Intelligence and Learning Systems Laboratory, National Technical University of Athens.