Testing whether a prediction transfers beyond its development data requires patients who were not part of that development. For the proposed NEO-GUARD34 work around NEAT, I made an independent patient-level dataset measuring the same outcome a condition for proceeding. On 21 August 2026, the public-data routes examined did not meet that condition, and I stopped the work.
I had limited the project to public data and ordinary CPU resources, without applying for patient data or expanding into new experiments. I directed the research question and stop criteria, with AI assistance in the review. The outcome at this stage was a decision about whether the project could proceed, not a performance result in an external patient group.
Separate rows are not necessarily separate patients
If one patient contributes several candidates or observations, randomly splitting table rows can leave information from that patient on both sides. That is different from testing on an unfamiliar patient. I therefore required patient-level separation and a dataset independent of the development lineage.
A separately named dataset would still be insufficient if it measured a different outcome. This outcome definition—the endpoint—had to match the claim being tested. Otherwise, a result on the second dataset would not provide the external check the project required.
Fixing the rules before looking at the evaluation
Changing a model or decision threshold after seeing evaluation results lets the evaluation set influence development. The proposed condition was to keep the holdout separate while model and threshold choices were made, and to fix the analysis procedure beforehand. Sealing the evaluation meant constraining that order of decisions.
The NEAT-only post hoc route and the alternative examined at the time did not supply the required independent patient-level holdout. I did not replace that condition with another split of the same material or further internal analysis. The recorded decision was NO-GO: do not proceed on those routes. It was not a finding that no suitable public dataset could exist anywhere.
What would follow if independent data became available
Recalculation, overlap between patients and candidates, and possible leakage from development into evaluation were still unfinished tasks. I did not reach the stage of completing those checks and validating performance externally.
A reason to reopen the project would be an actually available independent patient dataset with the same endpoint and usable access terms, not merely another dataset name. It would then need an evaluation kept separate from model selection. Those conditions had not been met in the routes examined on 21 August, which is where I stopped.