# ToxiCompass external validation summary v1

Frozen: 2026-08-02  
Website publication: 2026-08-10  
Status: completed with limitations

## Evaluation scope

This public summary records the frozen aggregate evidence from a post-hoc
robustness evaluation assembled and completed during manuscript revision. It
is not prospective blind validation. Stage 1 measures toxin-like discrimination
within a constructed external comparator cohort. Stage 2 is reported separately
as positive-only retained-label target/effect-family ranking. These are offline
Full-model results, not performance claims for the public Lite web service.

## Source cohorts

| Source | Role in the frozen evaluation | Retained records |
| --- | --- | ---: |
| ConoServer | Revision-period positive and Stage 2 cohort | 59 |
| ArachnoServer | Separate OOD/challenge positive cohort | 28 |
| Hemolytik 2.0 | Assay-scoped bioactive/non-hemolytic comparators | 521 |
| IUPHAR/BPS Guide to PHARMACOLOGY | Therapeutic-peptide and disulfide-rich comparator records | 21 |

The positive audit began with 114 records: 76 ConoServer Tier A candidates and
38 ArachnoServer high-confidence candidates of at most 50 amino acids. Exact,
accession, MMseqs90 cluster, and all-vs-all pairwise checks retained 87 records
and excluded 27 records with internal-overlap evidence.

## Stage 1

The formal frozen cohort contains 629 rows: 87 external positives and 542
operational comparators. The comparator strata are 521 assay-scoped
bioactive/non-hemolytic records, 20 therapeutic peptides, and one
disulfide-rich proxy. The 87-positive plus 87-comparator set is a separate
length-matched sensitivity analysis, not the formal headline cohort.

| Model | Role | AUROC (95% CI) | AUPRC (95% CI) | Sensitivity | Specificity |
| --- | --- | ---: | ---: | ---: | ---: |
| Primary Full ensemble | Frozen primary | 0.8972 (0.8549-0.9338) | 0.7428 (0.6557-0.8189) | 0.9540 | 0.4686 |
| Robustness Full ensemble | Robustness | 0.8721 (0.8243-0.9115) | 0.7256 (0.6444-0.8021) | 0.9195 | 0.5498 |

Primary intervals use test-sample percentile bootstrap (`B=10,000`, seed
`20260805`). The robustness-ensemble audit uses its frozen stratified bootstrap
(`B=2,000`, seed `20261801`).

## Stage 2

The Primary Full ensemble results are reported separately by external source.
Hit@k and mean reciprocal rank describe retained-label ranking. Micro-average
precision is a retained-vs-unannotated diagnostic because unretained labels are
not verified biological negatives.

| Cohort | n | Hit@1 | Hit@3 | MRR | micro-AP diagnostic |
| --- | ---: | ---: | ---: | ---: | ---: |
| ConoServer formal | 59 | 0.6271 | 0.8136 | 0.7432 | 0.6357 |
| ArachnoServer OOD/challenge | 28 | 0.4643 | 0.7143 | 0.6168 | 0.4460 |

## Interpretation limits

- The 542 comparator records are evidence-bounded operational comparators, not
  universally verified non-toxic peptides.
- The cohorts were assembled and evaluated during revision and are not
  prospective blind validation.
- The close non-toxic homologue stratum has no retained samples, so its
  false-positive rate is not applicable.
- Stage 2 is positive-only retained-label ranking, not calibrated mechanism
  truth, universal toxicity certainty, or ordinary multiclass classification.
- The selected Full results use query-only A3M fallback rather than database
  homolog-search MSA.
- Stage 1 and Stage 2 are not combined into one cascade headline metric.

See `external-validation-summary-v1.json` for machine-readable values and
provenance hashes, and `DATA_USE_NOTICE.md` for the distribution boundary and
source terms.