a 2024

Comparison of Large Language Models for Generating Contextually Relevant Questions

LODOVICO MOLINA, Ivo; Valdemar ŠVÁBENSKÝ; Tsubasa MINEMATSU; Li CHEN; Fumiya OKUBO et al.

Základní údaje

Originální název

Comparison of Large Language Models for Generating Contextually Relevant Questions

Autoři

LODOVICO MOLINA, Ivo; Valdemar ŠVÁBENSKÝ; Tsubasa MINEMATSU; Li CHEN; Fumiya OKUBO a Atsushi SHIMADA

Vydání

Proceedings of the 19th European Conference on Technology Enhanced Learning (ECTEL), 2024

Další údaje

Jazyk

angličtina

Typ výsledku

Konferenční abstrakta

Utajení

není předmětem státního či obchodního tajemství

Odkazy

URL, URL

Označené pro přenos do RIV

Ne

Organizace

Masarykova univerzita – Repozitář

ISBN

978-3-031-72312-4

DOI

https://doi.org/10.1007/978-3-031-72312-4_18

Klíčová slova anglicky

Generative AI; Question Generation; AI in Education
Změněno: 16. 9. 2024 00:50, RNDr. Daniel Jakubík

Anotace

V originále

This study explores the effectiveness of Large Language Models (LLMs) for Automatic Question Generation in educational settings. Three LLMs are compared in their ability to create questions from university slide text without fine-tuning. Questions were obtained in a two-step pipeline: first, answer phrases were extracted from slides using Llama 2-Chat 13B; then, the three models generated questions for each answer. To analyze whether the questions would be suitable in educational applications for students, a survey was conducted with 46 students who evaluated a total of 246 questions across five metrics: clarity, relevance, difficulty, slide relation, and question-answer alignment. Results indicate that GPT-3.5 and Llama 2-Chat 13B outperform Flan T5 XXL by a small margin, particularly in terms of clarity and question-answer alignment. GPT-3.5 especially excels at tailoring questions to match the input answers. The contribution of this research is the analysis of the capacity of LLMs for Automatic Question Generation in education.
Zobrazeno: 4. 5. 2026 12:08