PREreview de Training Large Language Models for Small-Molecule Design with Synthetic Task Scaling

Review of "Training Large Language Models for Small-Molecule Design with Synthetic Task Scaling"

Summary

This preprint asks whether LLMs can be post-trained for expensive, high-fidelity molecular design objectives by scaling training on cheaper synthetic tasks. The authors construct a two-tier hierarchy of synthetic lead-optimization tasks — cheap RDKit/OpenEye property optimization (Tier-1) and medium-cost Chemgauss4 docking (Tier-2) — and evaluate curriculum recipes on a held-out objective scored with SQM2.20, which is too expensive to train on directly. They report that curriculum-trained 35B models outperform a Tier-1-only baseline and several frontier models.

Objective

The introduction clearly explains the objective. It builds a gap-then-question structure: drug discovery requires searching a vast chemical space; scoring oracles range from cheap and low-fidelity to expensive and high-fidelity, and that cost limits search; LLMs are a good fit, but existing RLVR work trains only on cheap oracles, and training directly on expensive ones is impractical because online RL becomes too slow or produces stale rollouts. The objective follows as a research question — whether cheap synthetic tasks can substitute — phrased as "we investigate whether," which keeps it falsifiable. The phrase "too expensive to train on" names the constraint that makes the question non-trivial.

Methods — somewhat appropriate

The design is well matched to the question. Fixing all runs at 500 steps on identical hardware isolates the curriculum variable; T1(500) and T2(500) are the right anchors; PLINDER train/test splits prevent leakage; PoseBusters filtering, baseline normalization, zero-scoring of invalid rollouts, and the reward-weighting sweep in Appendix C.2 are all careful choices.

Four issues limit it:

1. The frontier comparison isn't like-for-like: Trained models had 500 steps of practice with the exact prompt format; frontier models had none. Table A3 shows Claude Opus 4.8 failing formatting in 39 of 200 rollouts and GLM-5.2 in 71, each scored as zero improvement. Part of the reported gap is formatting compliance rather than design skill, and no few-shot or format-adapted baseline separates the two.

2. Statistical resolution is thin for the finer claims: The top result is +3.88 ± 1.98 kcal/mol. Twelve recipes are compared, each with a single training seed, 40 systems, five rollouts, and no multiplicity correction. The ordering reshuffles substantially across the three reward-weighting settings (Figures 3, A3, A4), suggesting run-to-run noise is comparable to the differences being interpreted.

3. One base model, one metric: Everything starts from Qwen3.6-35B-A3B, so it is unclear whether the curriculum benefit is a property of the approach or of this checkpoint — particularly given the authors' own observation that the base model's low-halogenation tendency is amplified by training. SQM2.20 is the sole measure of binding, with no orthogonal check.

4. Minor asymmetry: The contains_scaffold tool is available at evaluation but withheld during training. This is explained, but it complicates the comparison.

Conclusions — somewhat supported

The central claim holds. Every curriculum model beats T1(500), and the T2(500) collapse into a constraint-only policy directly supports the argument that the curriculum is necessary. The efficiency findings (7.7 turns versus 13–18 for frontier models, with lower token counts) are large enough that noise doesn't threaten them. The authors are commendably honest where it costs them: they state plainly that no clear scaling law emerges, and they flag T1(250)–Mix(250) as an exception to their own conclusion about mixing.

Four claims reach past the data:

• "Surpasses frontier models" is never reconciled with the formatting failures the authors themselves document in Table A3.

• Recipe rankings such as "no clear benefit to mixing" are stated more firmly than single seeds and wide error bars allow.

• Chemical behavior is read as strategy: Exploration of worse intermediate designs is called "encouraging" evidence of non-greedy search, with no alternative considered. Anionic-group removal, worth a mean 26.1 kcal/mol, is treated as a learned design principle without testing whether it exploits an SQM scoring bias.

• The wet-lab extension in the conclusion is hedged with "potentially," but nothing in an all-in silico study speaks to experimental transfer.

Data presentation — somewhat appropriate and clear

Figure 1 carries the paper: the three-tier schematic with real prompt text makes the task hierarchy immediately legible, which matters most since task design is the contribution. Table 2 pairs the headline metric with turns, SQM calls, and constraint satisfaction. Table A3's failure breakdown is transparent. Figure A6 handles a wide dynamic range with a log scale and marks unavailable closed-source reasoning as "∼" rather than zero.

Limitations:

• Sign convention: Lower SQM means tighter binding, but "relative BA improvement" is constructed so higher is better, and Figure 5 labels improvements as "ΔSQM = +5.5 kcal/mol." Nearly every caption must restate "higher is better."

• Figure 3, right panel omits error bars "for clarity" while plotting differences comparable in size to the SEs shown in the left panel, making separation look cleaner than it is.

• Figure 2, right panel overlays five curve pairs in similar greens and oranges; distinguishing T1(350)–T2(150) from T1(350)–Mix(150) is difficult.

• Color accessibility: The green/orange trained-versus-zero-shot encoding (Figures 3, A3, A4, A8) presents the standard deuteranopia problem. Figure 7 uses a red-green diverging heatmap and suppresses values below 10%, leaving most cells readable only by color.

• Crowding: Figure 6's leader lines converge on small labels near the origin; Figure A10's rotated labels are ordered by value rather than recipe family.

Discussion — somewhat clear

The mechanistic explanations are strong. Appendix C.1 gives the T2(500) collapse a reward decomposition showing the policy saturating constraints while making no binding progress, which explains why the curriculum is needed rather than merely asserting it. Design choices come with reasoning: the tool-penalty warmup is justified by what failed without it, and withholding the scaffold tool during training by the mode collapse it caused. Next steps are concrete — turn-level rewards, supervised finetuning, on-policy distillation.

The conclusion, however, is a single substantive paragraph that mostly recapitulates. It never revisits the paper's own complications — the formatting failures, the reward-weighting sensitivity, the absent scaling law — which remain in appendices. There is no limitations section: single base model, single scoring function, one seed per recipe are nowhere acknowledged as bounding the claims.

Contribution — somewhat likely to advance knowledge

The work addresses a real bottleneck. SQM takes tens of minutes per evaluation and FEP takes GPU-hours, which has confined RLVR molecular design to cheap rewards. Demonstrating transfer from cheap synthetic tasks to an expensive held-out objective offers a concrete route around that constraint, and the pattern should generalize beyond chemistry.

The T2(500) collapse is the most valuable single finding — a mechanistic negative result that converts "curriculum helps" into "curriculum is necessary here." The efficiency result deserves more attention than it receives. The task-construction recipe (200,000 verifiable tasks from 116 descriptors, plus structural textification) is reusable.

Against this: curriculum learning and synthetic data scaling are established, so the novelty is the application rather than the principle. The frontier comparison carries much of the rhetorical weight but is confounded. And whether the models learned chemistry or learned SQM's biases remains unresolved — the question that determines whether any of this transfers to real campaigns.

Language

No editing needed. The writing is clean, terminology is defined at first use, and notation is consistent. The mixed polarity of the SQM sign convention is a presentation choice rather than a language problem, though it does force repeated clarification.

Competing interests

The author declares that they have no competing interests.

Use of Artificial Intelligence (AI)

The author declares that they used generative AI to come up with new ideas for their review.