Prospective evaluation of PlasmidCall on 150 bacterial genomes across six taxa: assemblies, reference labels, classifier calls and evaluation evidence
Year: 2026
Type: dataset
Abstract: What was done. A total of 150 bacterial genomes were assembled de novo with Unicycler from public paired-end Illumina reads, 25 from each of six taxa: three species (Klebsiella pneumoniae, Enterococcus faecium and E. faecalis) and three genera (Enterobacter spp., Citrobacter spp. and Serratia spp.). Each isolate was paired with the complete reference genome (NCBI assembly level Complete Genome) deposited for the same BioSample, from RefSeq for 149 isolates and from GenBank for one. All 19,320 assembled contigs were aligned to that reference with minimap2, and the 9,784 contigs of at least 1 kb were eligible for a label. Of these, 9,371 resolved to a chromosome (7,130) or a plasmid (2,241) of the reference, as designated in its NCBI assembly report, and 413 remained unresolved (300 aligned to both classes without the required margin and 113 reached neither). Resistance determinants were annotated on the same contigs with AMRFinderPlus 4.2.7 (database 2026-08-07.1), so each determinant on a resolved contig can be placed on the chromosome or on a plasmid of the reference. Evaluation. The chromosome and plasmid labels were then used to score a prospectively sealed evaluation whose predictions were frozen before any label for these genomes was constructed. PlasmidCall, which takes the calls of 12 third-party classifiers as input, was compared with those 12 classifiers and with three baselines: a strict majority of the 12, at least one of the 12 calling plasmid, and every contig called chromosomal. Before the freeze, the candidate prediction table was rebuilt from its hashed inputs and matched byte for byte. The frozen table, projected back onto the candidate's columns by the freezing script and by a separately written projection script, also matched byte for byte. The same operator ran all of these checks. Principal derived result. Of the 635 resolved contigs that carry at least one antibiotic resistance determinant (AMRFinderPlus core AMR genes, efflux excluded), from 145 of the 150 isolates, 266 (41.9%) are plasmid-derived. Across the six taxa the proportion ranges from 57.4% in Citrobacter spp. (54 of 94 contigs) to 7.1% in Serratia spp. (4 of 56). Forty gene groups occur on at least 10 of these 635 contigs; a group is an AMRFinderPlus gene symbol with its allele number removed, so sul1, sul2 and sul3 form one group. Three of the 40 are biocide or metal determinants (qacEdelta, ssmE and silA). Of the other 37, 15 have at least 90% of their contigs plasmid-derived, 12 have at most 10%, and 10 fall in between. These groups are not all independent: seven of the 15 are members of the vanA gene cluster in the same 14 E. faecium isolates, and aph(3'')-Ib and aph(6)-Id sit on the same 11 contigs. These counts combine the AMRFinderPlus annotation with the reference labels; no plasmid classifier output enters them. Models and code. The index classifier was PlasmidCall v1.2-General at its frozen threshold 0.9285. It was evaluated alongside PlasmidCall v1.1, at thresholds 0.9524 and 0.9605, and a frozen router that assigns each contig to one of the two models; the frozen scores and calls of all three are in Parts 1 and 3. This deposit holds no model files. Both frozen models and the code that scored them are in the PlasmidCall software release v1.1.0 at https://doi.org/10.6084/m9.figshare.34018380.v1 (source: https://github.com/piranfar/PlasmidCall); the two model files there have the SHA-256 digests recorded at the prediction freeze. Classifier settings. Three of the classifiers take a species setting, and a fourth, gplas2, classifies from PlasmidEC's output and so inherits its setting. RFPlasmid was run with its Enterobacteriaceae model, and PlasmidEC and PlaScope with their Escherichia coli setting (for PlaScope, the E. coli index it ships with), for all six taxa. None of the six taxa is E. coli, and the two enterococcal taxa, which hold 3,914 of the 9,371 resolved contigs, lie outside the Enterobacteriaceae. A later, post hoc re-run of RFPlasmid on the 50 enterococcal isolates, changing only its species argument to Enterococcus, raised its precision on their contigs from 0.532 to 0.702. With those calls substituted into the frozen table, its precision over the whole cohort rises from 0.666 to 0.783 and its F1 from 0.743 to 0.812, with recall almost unchanged (0.841 to 0.843). The rows with a higher F1 than PlasmidCall v1.2-General then rise from six to seven: six third-party classifiers instead of five, plus the majority baseline. PlasmidCall's own frozen scores were computed from the same panel settings and were not recomputed. At their default operating points, no third-party classifier reaches a precision of 0.95 before or after this correction; the highest is Platon at 0.936. An Enterococcus index for PlaScope was built after the evaluation and is deposited, with its build recipe and a comparison run, at https://doi.org/10.6084/m9.figshare.33990655; that index contains the reference genomes of these 50 isolates and should not be used to re-score this cohort. PlasmidEC's Enterococcus setting could not be measured, because the mlplasmids package it needs could not be installed in our container, so PlasmidEC and gplas2 have only their frozen figures. Contents. Part 1, PlasmidCall_evidence_tier1_execution.tar.zst: parsed classifier calls, execution receipts, resistance-gene annotation, provenance, the frozen prediction table, and in env/ the execution scripts as they stood on the execution host (read downsampling, isolate pairing, assembly acceptance, panel execution and parsing), with 11 superseded copies kept with a .pre_ suffix. Part 2, PlasmidCall_evidence_tier2_assemblies.tar.zst: the 150 genome assemblies with their assembly graphs and logs. Part 3, PlasmidCall_evidence_tier3_evaluation.tar.zst: the joined reference and prediction table, which gives the aligned fractions behind each label, the per-contig error catalogue, the characterisation of the unresolved contigs, the sensitivity analysis, the tool-output accounting and a copy of the frozen prediction table. Part 4, PlasmidCall_published_results.tar.zst: Tables 1 to 5, Supplementary Data 1 to 11 and the source data for Figures 1 to 8 of the manuscript version of 24 August 2026, with the records behind them (canonical numbers and tables, the resistance-gene context record, the claim audit, the claim-to-evidence matrix, the artefact inventory, and the intended-use and limitations statements). What this deposit does not contain. The reads are cited by run and sample accession, with the MD5 checksum of every file as verified at acquisition, and each reference genome by its assembly accession, in supplementary_data/Supplementary_Data_1.tsv (Part 4); neither is redistributed here. The native, unparsed outputs of the classifier panel are retained by the author and are available on request. The frozen evaluation does not need them: it uses the parsed calls and the frozen prediction table in Part 1 and the joined table in Part 3. The scripts that built the reference labels and computed the evaluation (truth acquisition, mapping, joining and metrics) are not included; the SHA-256 of the mapping script is recorded in records/PLASMIDCALL_CANONICAL_NUMBERS.json (truth_mapper_sha256). The post hoc RFPlasmid re-run is not in this deposit either. Integrity. MANIFEST.sha256 gives the SHA-256 of the four archives. Its paths carry a parts/ prefix, so put the archives in a folder named parts/ beside MANIFEST.sha256 before running sha256sum -c MANIFEST.sha256, or compare the digests directly. Zenodo also lists an MD5 for every file. Parts 1 and 2 are the archives built on the execution host, renamed but byte for byte unchanged, and their digests match the values recorded when they were downloaded from it. Corrections to the deposited files (version 1.0.0). README.md: "six genera of clinical importance" should read "six bacterial taxa of clinical importance, three defined at species level and three at genus level"; "57.5% in Citrobacter spp." should read "57.4%" (54 of 94 contigs); "MANIFEST.sha256 carries a SHA-256 for every file" should read "for each of the four archives", and the first line of MANIFEST.sha256 overstates its coverage in the same way; "match the values recorded at the results freeze" should read "match the values recorded when they were downloaded from the execution host"; and of the six files in Part 3, only the joined table and the frozen prediction table have digests recorded in Part 4. Its statement that all 19,320 assembled contigs were aligned is correct: the 1 kb floor was applied after alignment, to decide which contigs received a label and were evaluated. Part 4: the vanA-, vanB- and vanD-type entries (98, 14 and 5) in Table 5, the Figure 8 source data, records/PLASMIDCALL_ARG_CONTEXT_BIOLOGY.json, records/PLASMIDCALL_CANONICAL_NUMBERS.json (argctx_vanA, argctx_vanB and argctx_vanD), records/PLASMIDCALL_CANONICAL_TABLES.json and claim C30 count gene occurrences in every count column, not contigs; these genes lie on 28, 2 and 2 contigs in 14, 2 and 1 isolates. Claim C27 in the claim records gives 57.5% for Citrobacter spp., and tables/Table5_arg_genomic_context.md gives 0.575; the correct values are 57.4% and 0.574. The calibration intercept and slope in records/PLASMIDCALL_CANONICAL_NUMBERS.json do not reproduce: recomputed from the frozen scores, they are 0.0723 and 0.9697 for v1.2-General (recorded as 0.5415 and 0.0818) and 0.9016 and 0.8195 for v1.1 (recorded as 0.6456 and 0.0888). The Brier scores and expected calibration errors do reproduce. Claim C03 holds only at each classifier's default operating point: after the freeze, a retuned threshold gave Plasmer a precision of 0.9547 at recall 0.7046. Claim C29 and the reading of the family distribution count the genes of the vanA cluster as separate families (see Principal derived result). Earlier versions of this description stated that only the contigs of at least 1 kb were aligned, and their erratu
Author: Piranfar, Vahhab
Language: English
Subfield: Molecular Biology
Domain: Life Sciences
Open Access status: green
OpenAlex ID: W7204099282