Year: 2025
Type: conference-paper
Abstract: Large language models (LLMs) are increasingly becoming valuable to corporate data management due to their ability to process text from various document formats and facilitate user interactions through natural language queries.However, LLMs must consider the sensitivity of information when communicating with employees, especially given access restrictions.Simple filtering based on user clearance levels can pose both performance and privacy challenges.To address this, we propose the concept of sensitivity awareness (SA), which enables LLMs to adhere to predefined access rights rules.In addition, we developed a benchmarking environment called ACCESS DENIED INC to evaluate SA.Our experimental findings reveal significant variations in model behavior, particularly in managing unauthorized data requests while effectively addressing legitimate queries.This work establishes a foundation for benchmarking sensitivity-aware language models and provides insights to enhance privacy-centric AI systems in corporate environments.
Authors: Dren Fazlija, Arkadij Orlov, Sandipan Sikdar
Institutions: Leibniz University Hannover, L3S Research Center
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
SDG: Reduced inequalities
Open Access status: gold
OpenAlex ID: W4412888002