Intent Recognition Using Distilbert and Language Models
Year: 2024
Type: article
Abstract: Intent classification in Natural Language Processing involves identifying the intention of the user based on their input/interaction with an interface. This can be in a natural usage setting (voice assistants) or an interaction between users, customer service personnel, or agents (in a large organization). This paper aims to study the problem of intent classification using language models (like DistilBERT) and Large Language Models (Phi2 and LLAMA3) on open-source datasets available in banking, travel, small talk, office, etc. This is a challenging problem as there is a need for more available data that spans various domains. Identifying the user’s intention is sometimes tricky as the classification changes based on the context (and ambiguity in natural language). Apart from the given intents, the data poses a challenge when a user presents the interface with out-of-scope queries (for which the models aren’t trained). We test this on transformer-based approaches like DistilBERT and LLMs (like Phi2 and LLAMA3). We find that DistilBERT, with fewer parameters, trains faster and runs inference faster than LLAMA3 and Phi2 (with PEFT-LoRA). We also find that the results from DistilBERT are much better than those of the language model-based approaches for the Banking-77 and CLINC-150 datasets. These two datasets cover various domains (as mentioned above).
Author: Researcher
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6893026245
FWCI:
Cites:
Cited by:
Related to: