Investigating the Use of Intelligent Tutors Based on Large Language Models: Automated generation of Business Process Management questions using the Revised Bloom's Taxonomy
Year: 2024
Type: conference-paper
Abstract: The construction of assessment artifacts is a complex task, since generating appropriate assessments manually requires in-depth knowledge of both the area to be assessed and the cognitive processes involved in learning. The use of Large Language Models (LLMs) as the basis for the operation of Intelligent Tutoring Systems can assist in this task. This work experiments with the GPT-3.5-Turbo and LLama-2 LLMs as a source of automatic generation of assessment questions. The experiment was carried out using Prompt Engineering techniques to generate questions for the Business Process Management (BPM) discipline. From the experiment, it was possible to observe that both models are capable of generating questions appropriate to the BPM context. It was also identified that, when it received the context and the model of the question to be generated, the LLama-2 model produced questions more appropriate to the desired cognitive level, while the GPT-3.5-Turbo model received only the context and produced a similar response.
Institution: Universidade Federal do Rio Grande do Sul
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: gold
Award: G29316538
OpenAlex ID: W4404839736