HumanandLLM-based Assessment of Teaching Acts in Expert-led Explanatory Dialogues
Understanding the strategies that make expert led explanations effective is a core challenge in didactics and a key goal for explainable AI. To study this computationally, we introduce ReWIRED, a large corpus of explanatory di alogues annotated by education experts with f ine-grained, span-level teaching acts across f ive levels of explainee knowledge. We use this resource to assess the capabilities of modern language models, finding that while few-shot LLMsstruggle to label these acts, fine-tuning is a highly effective methodology. Moving be yond structural annotation, we propose and val idate a suite of didactic quality metrics. We demonstrate that a prompt-based evaluation us ing an LLM as a “judge” is required to cap ture how the functional quality of an explana tion aligns with the learner’s expertise– a nu ance missed by simpler static metrics. Together, our dataset, modeling insights, and evaluation framework provide a comprehensive methodol ogy to bridge pedagogical principles with com putational discourse analysis.