Work

A Novel Adversarial Augmentation-based Domain LLM Framework for Perceiving IoT Service States

Year: 2025

Type: article

Abstract: Accurately perceiving service states of the Internet of Things (IoT) is crucial for maintaining system stability and long-term sustainability. Recently, Large Language Models (LLMs) have emerged as a novel technological advancement, offering unprecedented possibilities in various domains due to their advanced comprehension capabilities. However, the limited data availability problem poses a challenge to the integration of domain knowledge into LLMs and the enhancement of their ability to perceive service states. In this article, a novel adversarial augmentation-based domain LLM framework for IoT services is proposed to solve this problem. We first construct an LLM fine-tuning dataset through an elaborate hierarchical prompt template, which integrates task-specific instructions, heterogeneous service data, and statistical features to the service contextual description. Then, inspired by using specialized compact small-models to enhance the capabilities of LLMs, we design two dedicated domain-specific small-models, including a service topology prediction model and a running status prediction model, to capture evolution patterns of services from different perspectives. On this basis, we design an LLM fine-tuning framework based on the domain adversarial distillation, which comprises an LLM parameter tuning module and a small-model adversarial distillation module. The former leverages the Low-Rank Adaptation (LoRA) algorithm to fine-tune the LLM. The latter distills the domain knowledge of small-models into the LLM through aligning the latent vector distributions of the LLM with those of small-models. Following the adversarial training, the LLM will be equipped with domain capabilities of small-models. Experimental results on two datasets demonstrate the effectiveness of our LLM framework.

Source: ACM Transactions on Internet of Things

DOI: 10.1145/3762668

Authors: Peng Qi, Zuodong Jin, Dan Tao, Ruipeng Gao

Institution: Beijing Jiaotong University

Language: English

Topic: Domain Adaptation and Few-Shot Learning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

SDG: Industry, innovation and infrastructure

Open Access status: bronze

Funders: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities