Learning by Assessing AI: Peer Assessment of Large Language Models in an Introductory Logic Course

This paper investigates how peer assessment as a learning strategy can be adapted to educational settings involving large language models (LLMs). In a redesigned introductory logic course, students were trained to evaluate the ability of an LLM to analyze and prove syllogisms. The approach integrates assessment as learning with a model of human–AI interaction in which students act as critical evaluators of AI outputs. Students developed and compared prompts designed to produce correct logical analyses, and group performance was measured based on their ability to identify the strongest prompts. Results show that student groups were able to identify high-quality prompts, achieving a mean score of 86.1%, significantly above chance, with a large effect size (1.85). These findings suggest that students can be meaningfully positioned as assessors of AI-generated reasoning, providing a viable pedagogical approach for integrating generative AI into logic education. The observed performance is consistent with learning in formal reasoning.