TRIASE-SLO: Risk-Gated and SLO-Aware Routing Between FP16 and INT4 Language-Model Workers for Radiology Impression Serving
Year: 2026
Type: article
Abstract: Language models can draft radiology impressions, but a 4-bit copy of a model does not always reproduce the output of its full-precision version. TRIASE-SLO routes each request between FP16 and AWQ INT4 copies of one LoRA-adapted Qwen2.5-3B-Instruct model. A frozen rule-based hazard score computed from the FINDINGS text sends every request scoring at least 0.5 to FP16, and a secondary latency-aware score places the remaining requests. We evaluated the design in a locked confirmation in which every arm reserved the same two A100 MIG slices: four arms, seven loads and 12 paired deployment blocks of 450 previously inspected test reports (151,200 requests, none failed). The gate improved agreement with FP16. With full TRIASE-SLO routing, agreement with finding-bearing FP16 references was 99.11–99.49%, compared with 66.15–82.31% under hazard-blind routing, while 34–48% of requests still used INT4. A consistent serving-efficiency advantage was not demonstrated: full routing did not improve tail latency or throughput over two FP16 workers (median p95 change +0.33%; slower at C = 2 after Holm correction), and a consistent speed advantage of INT4 appeared only under hazard-blind routing. The demonstrated benefits of the INT4 copy are a smaller checkpoint and a larger KV-cache capacity. The incremental value of the score remains uncertain, because its prespecified C = 8 throughput benefit over gate-only routing (+1.86%, 95% CI −3.27% to +7.27%) was not confirmed. No INT4-using arm met the zero-drift engineering floor. All quality measures are rule-based FP16-consistency proxies from one public dataset, not evidence of clinical correctness or safety.
DOI: 10.62411/jcta.18198
Authors: Junta Zeniarja(Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft), Guruh Fajar Shidik(Conceptualization, Funding acquisition, Methodology, Project administration, Resources, Supervision, Validation, Writing – review & editing), Heru Suhartanto(Conceptualization, Formal analysis, Methodology, Resources, Supervision, Writing – review & editing), Aris Marjuni(Conceptualization, Methodology, Resources, Supervision, Writing – review & editing)
Institutions: Universitas Dian Nuswantoro, University of Indonesia
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7236071528
