Year: 2025
Type: dissertation
Abstract: Modern machine learning methods are actively applied across various fields of science and technology, delivering high-precision solutions and enabling the automation of complex processes. However, one of the key challenges in the development of these technologies is the need for selective removal of information from trained models. This necessity arises from both regulatory requirements for personal data protection—such as the General Data Protection Regulation (GDPR)—and practical needs, including model adaptation to changing conditions and the correction of errors in training data. In response to these challenges, the field of Machine Unlearning has emerged and gained momentum in recent years. It offers methods for correctly removing the influence of specific data without the need to retrain the entire model. The primary difficulty of machine unlearning lies in the fact that knowledge in large language models is not localized in specific parameters but is instead distributed throughout the model’s architecture. This makes targeted data removal a non-trivial task, requiring the development of specialized algorithms. This practice explores modern unlearning methods, including Knowledge Editing and Simplified Negative Preference Optimization (SimNPO), which aim to remove information with minimal impact on the overall performance of large language models. The relevance of this work stems from the pressing need to develop effective machine unlearning algorithms that meet regulatory requirements, protect models against attacks such as data poisoning, and enable adaptation to continuously evolving datasets. Currently, most existing methods either require full model retraining—which is computationally expensive—or fail to guarantee precise data removal. The development and evaluation of more efficient knowledge removal strategies is a critical task for ensuring the safety and reliability of large language models. Object of the study: Machine unlearning methods for large language models. Subject of the study: Principles and algorithms for removing information from trained models without full retraining. Purpose of the practice: To study, analyze, and implement machine unlearning methods using text-based models as examples. To achieve this goal, the following tasks must be completed: Analyze existing machine unlearning methods, including exact and approximate unlearning approaches. Study modern techniques for removing information in large language models (LLMs), including Negative Preference Optimization (NPO) and Simplified Negative Preference Optimization (SimNPO). Compare unlearning methods and knowledge editing techniques in terms of deletion accuracy and model performance retention. Evaluate the computational costs of different approaches and their applicability to real-world machine learning tasks.
Author: Чурсин, А. Д.
Language: Russian
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7117205217