Optimising helpdesk efficiency: Separating contextually significant emails from low-information content using an entropy-based threshold
Year: 2026
Type: dissertation
Abstract: Real-world interactions consisting of a transactional nature can be inherently noisy. As a result, comprehension of the underlying objective of an exchange can be difficult to establish, especially when multiple topics are being discussed. From this, uncertainty can emerge within the discourse taking place.Uncertainty is prevalent within many task-oriented dialogue systems. A helpdesk for example, will elicit a substantial level of uncertainty by its very design, in the process of users seeking assistance. An advisee often does not have a comprehensive knowledge of the issue they are experiencing, hence the need for some form of expert intervention. The subsequent exchange can be noisy or lacking in information that is contextually significant. As a result, the level of contextual comprehension can suffer.This research sets out to explore whether a distinction can be made between those messages that are contextually significant and those that are considered as irrelevant to the task objective. The statement “I’m having printer issues” is more useful to a helpdesk advisor than “Where can we meet?” for categorisation purposes and should therefore carry more weight. We assess the impact that this perceived ambiguity has on the performance of a classification model and how it can be mitigated accordingly.Shannon’s entropy is a well-known metric within the field of information theory. Through the application of entropy in conjunction with the probability distribution of a model’s topic prediction for a message, a perceived measure of certainty can be derived. This metric is applied to messages within a locally acquired university helpdesk dataset to assess the impact that uncertainty can have on the process of categorising user issues. This metric is also applied in conjunction with a resampling technique to increase the exposure of useful instances within the dataset and reduce the exposure of low-information messages.From the findings derived from both literature and exploratory analysis, an approach was developed that separates those messages which can be considered useful from those that are not. It was found that through this approach, classification performance improved significantly.
Author: Fellows, Ryan
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7204354529