EN | 한국어
A prediction score of 50 does not mean that half the answers are correct. In the Virtual Embryo Challenge (VEC), a specified baseline prediction is assigned 50 points. Reading the final number without knowing how the underlying measurements were rescaled can give the wrong impression of a result.
I built a Korean score explainer, introductory guide and practice notebook to make that conversion visible. The browser tool needs no installation or login: readers can change synthetic numbers and follow their transformation into each metric’s contribution to the total.
What is a virtual embryo being asked to predict?
VEC uses measurements from observed stages of mouse development to predict other embryonic states. Its three tasks cover gene-expression distributions over time (T1), expression together with cells’ 3D locations (T2), and changes following a gene knockout (T3). Gene expression here is a measurement of how active each gene is within a cell.
Evaluation asks several questions rather than checking a single answer. Did the prediction recover the genes that changed, get the direction right, and reproduce the proportions of different cell states? The resulting metrics have different units and directions of improvement. The official rules therefore convert each one to a common scale, called skill, using two reference values.
Why the floor is worth 50 points
The floor is the result of a specified simple prediction: carrying the last observed state forward for T1 and T2, or returning the matched wild type unchanged for T3. The ceiling is estimated by splitting the target data in two and scoring one half against the other. It represents an attainable level given the data’s variability, not a perfectly known truth.
For a value worse than the ceiling, let d be its distance from the ceiling and d_floor the floor-to-ceiling distance. The conversion is skill = d_floor / (d_floor + d). At the floor the distances are equal, giving 1/2; at the ceiling skill is 1. Twice the floor distance gives 1/3. Following the official scoring description, the tool assigns skill 1 to a value at or better than the ceiling, then combines the skills with fixed weights on a 100-point scale.
I laid out raw value → required transformation → skill → contribution to the total instead of displaying only a final score. The table, curve and contribution bars show where a change in one input affects the calculation.
When −2 and +2 receive the same score
TSR (scale_log_ratio), a measure of growth-scale discrepancy, is scored by distance from zero rather than by sign. In the tool’s synthetic example, a raw value of −2, floor of −4 and ceiling of +1 become 2, 4 and 1 after taking absolute values. The ceiling distance is 1 and the floor distance is 3, so skill is 3/4. Changing the raw value to +2 gives the same result.
In T2, TSR carries one third of the tissue-shape-and-growth group’s 25% weight. Its contribution in this example is therefore 100 × 1/4 × 1/3 × 3/4 = 6.25 points. These are teaching numbers, not embryo measurements or leaderboard results. Real inputs require metric, floor and ceiling values from the same evaluation conditions.
PSS (severity_slope), the response-magnitude metric, is also scored on its absolute value, but the scorer already returns a logged value. Entering the regression slope itself or taking a second logarithm would change the calculation. I made invalid reference values stop the calculation, including cases where the floor and ceiling become equal after the absolute-value transformation.
Missing values and interval endpoints
An empty input and an explicitly missing metric take different paths. An empty field waits for an input; an explicit NaN receives skill 0 without redistributing its weight. If only T3’s PSS is missing while every other metric reaches its ceiling, the total is 75, because PSS retains its 25% share.
The tool also includes a small exercise on the stage boundaries in rule 10. For T1 and heart-extrapolation T2, the restricted interval (E9.5, E13.5] excludes E9.5 but includes E13.5; E denotes an embryonic stage. This checks only the selected numerical interval. A stage outside it is not permission to use or redistribute a dataset: genotype restrictions, source disclosure and data or model licences still need separate checks.
What I made public
I separated the calculation core from the interface and provided a Node verification script that loads the same core and compares it with independently specified constants and fractions. Synthetic cases cover sign transformations, fixed weights, missing metrics, invalid reference values and interval boundaries. They test the conversion code, not biological prediction performance or users’ learning outcomes.
The repository also contains the Korean guide and an output-free notebook for practising with the official local scorer, veckit. I excluded biological data and recorded execution outputs from the public package. This is an unofficial community resource, not an organiser-reviewed scorer; the source and test map links the explanations and implementation to their references.
Try it: Korean score explainer · source repository