Work

Checking Ai Output In Police Open-Source Investigations: A Step-By-Step Methodological Review

Year: 2026
Type: preprint
Abstract: This methodological review asks, for each step of a police open-source investigation, what can go wrong and what can go unexpectedly right when AI tools are used, and which checks protect the investigation against the first while keeping it open to the second. It reviews studies, tests, standards and EU law across four steps: collection, processing, analysis and verification, and reporting. Reported performance varies at every step, and a published figure does not show whether the tool will do worse or better on a new case. In this paper's reasoning, based on info-gap decision theory, checks that rely on the tool, such as its confidence, a second model's agreement or a reviewer who sees only the AI output, weaken when the tool is worse than believed. By the same reasoning, checks that bring outside information, such as the original file, an independent source or a second method, hold up better, since their judgement does not rest on the tool's output. The tool may also perform better than believed, including finding what nobody asked for, and the investigation keeps that gain if AI output enters first as a lead and becomes evidence only after a check that brings outside information. The review proposes that units place such checks before AI output grounds an arrest, a warrant, a data request, a publication or a court filing, and record every AI use so that others can check the work. None of these checks has been tested in police casework.
Source: CrimRxiv
Language: English
Subfield: Law
Open Access status: green
OpenAlex ID: W7225987423
Cites: 13
Cited by:
Related to: 10