HARD PROBLEMS · 2026 · EXTERNAL-VALIDATION NO-GO · NO CLINICAL PERFORMANCE CLAIM
NEAT/NEO-GUARD34 stopped because of a data condition, not because an internal metric needed one more round of tuning. There was no suitable independent patient-level dataset with the same endpoint, so a sealed external validation could not be run. External validation of the flagship claim remains zero.
No patient information, internal performance figure or treatment implication appears here. The useful record is the decision rule: define the dataset that must exist before allowing development results to become a clinical claim.
Development data
internal work
Independent patient data
no suitable set
Sealed validation
could not run
External performance checks
0
A public input is not automatically an external validation set
Repositories and published datasets may be available while still overlapping the development lineage or using a different endpoint. Re-splitting the same source does not necessarily separate model choices from evaluation.
The proposed gate required patient-level independence, a matching outcome definition and a prespecified sealed analysis. No dataset meeting those conditions was identified. Internal split results were not used to fill that gap.
Promising development numbers could not overrule the stop condition
Every additional round of tuning uses information from the development cohort. Numbers may improve while uncertainty about another patient population grows. Without an independent cohort, a more attractive internal result would still leave the main question unanswered.
The internal record covers reproduction, splitting, overlap and leakage controls, and the stop decision. Exact source commits, reproduced metrics and a frozen leakage audit would need to be re-established before any technical release. No clinical performance or treatment-effect statement is supported now.
When there is no independent patient cohort to test, a higher internal score is not a substitute for external validation.
The restart condition is stricter than finding another dataset name
A legitimate restart needs patient-level data independent of the development lineage, the same endpoint, and usable ethical and licensing terms. The analysis plan and metrics must then be frozen before a single external evaluation.
Until that run exists, the model cannot be described as clinically validated or useful for treatment choice. The verified outcome is only that the project failed its external-data gate and stopped.
Project record
- Period: 2026
- Activity: reproducibility review, patient-level split and overlap/leakage design, external-validation stop rule
- Internal evidence: reproduction, split and stop-decision materials
- External validation: no sealed patient-level validation of the flagship claim
- Final outcome: NO-GO because no suitable independent dataset was available
- Not established: performance in an outside cohort, clinical generalization or treatment utility