Work

Product-Specific Action Fidelity and Short-Term Repeatability of Consumer LLM Platforms for Psoriasis Biologic Self-Management: Reproducibility Package

Year: 2026
Type: dataset
Abstract: This repository contains the frozen reproducibility package for the study “Product-Specific Action Fidelity and Short-Term Repeatability of Consumer LLM Platforms for Psoriasis Biologic Self-Management: A Repeated Benchmarking Study.” The study evaluated four consumer-facing large language model platforms using 32 neutral product-specific scenarios and 16 paired misleading-premise prompts for COSENTYX, TALTZ, BIMZELX, and TREMFYA. Formal queries were conducted in three planned rounds, yielding 576 planned query instances, of which 572 were eligible for analysis. The package contains the frozen protocol and statistical analysis plan; scenario and prompt banks; source-action and provenance metadata; query-level metadata; final adjudicated ratings; derived analysis datasets; analysis/reproduction code; amendment and governance records; and file-integrity manifests required to reproduce the reported numerical results. The primary endpoint was scenario-weighted product-specific action concordance (PSAC). Secondary analyses included safety-critical concordance, misleading-premise robustness, prespecified contrastive product-rule discrimination, and short-term operational repeatability. Official FDA/manufacturer source documents are cited and mapped through provenance metadata rather than redistributed where redistribution rights are uncertain. Historical source files and governance records are retained without retrospective alteration. Version 1.0 represents the submission-freeze reproducibility release.
Language: English
Field: Medicine
Open Access status: green
OpenAlex ID: W7221149005
Cites:
Cited by:
Related to: 10