Preprint Open access
Spec2Game: Can LLMs Generate Complete Playable Games from Detailed Specifications?
Generating an executable program does not necessarily mean that it correctly implements the behavioral requirements specified in natural language. To evaluate large language models' ability to realize detailed specifications as complete interactive programs, we introduce Spec2Game, a benchmark that requires models to g …