Growth is not automatically progress
Adding nodes and edges can increase memory traffic, routing cost, and optimization difficulty. A network that grows quickly is useful only if the added capacity solves a problem and its per-step cost remains measurable. Our proposal starts with a fixed active budget rather than unrestricted graph expansion.
Cells that can express more
A cell could use a gated rational update: a learned numerator divided by a positive, bounded-away-from-zero denominator, followed by a contracted residual. This connects to the CFENG character model, but graph stability would need its own experiments. A compact cell is not evidence of a fast graph implementation.
Learn which connections are useful
Instead of refreshing every edge every step, keep a packed active neighborhood and evaluate candidate edges on a slower schedule. Gradient-based scores may identify useful communication paths. A control graph with static edges is essential: otherwise routing overhead can masquerade as architectural progress.
Split responsibility, preserve behavior
When a cell is persistently overloaded, a proposed growth operation would split its responsibility while initially preserving the network output. New capacity could then specialize. Maintaining that initial function, optimizer state, and balanced routing is a central technical challenge.
The decisive experiment
Compare dense, static sparse, and adaptive systems under matched hardware, wall time, active parameters, and data. Measure held-out quality, actual examples per second, memory, growth overhead, and seed variance. The proposal is promising only if those measurements support it. No benchmark result is claimed yet.