Bridging the Deployment Gap: Lessons from NLP and Predictive Modeling in Production Healthcare and Insurance Systems
Year: 2026
Type: preprint
Abstract: Getting a machine learning model to work well on a benchmark is hard. Getting it to work in a hospital or an insurance company, where the data is messy, the stakeholders are skeptical, and being wrong has real consequences, is a different problem entirely. This paper reflects on two systems built and deployed in production: a compliance-sensitive NLP based document retrieval system for a large enterprise client, and a suite of predictive models for healthcare and insurance analytics across fifteen enterprise clients. We present empirical results comparing three post-hoc explanation methods (Attention, LIME, SHAP) on BGE-M3 using MultiNLI. Attention achieves 89.6% audit readiness, SHAP 77.1%, and LIME 45.8%. No method reliably satisfies compliance deployment requirements across all examples.
Author: Jagdish Aslesha Desetty
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7162117173
Cites:
Cited by:
Related to: