Introduction A central challenge in neuroscience is to understand how collective computations arise from the coordinated activity of many interacting units. High order interactions (HOIs)—statistical dependencies not reducible to pairwise relations—offer a principled way to quantify such emergent structure. Yet, the mechanisms that generate HOIs and their relationship to the geometry of population dynamics remain poorly understood. Here, we study how HOIs self organize in recurrent neural networks (RNNs) trained on cognitive tasks of varying complexity, and we identify a general link between informational structure and the dimensionality of the underlying dynamical trajectories.
Methods Continuous‑time RNNs were trained on four tasks spanning a range of cognitive demands: Go/NoGo, Negative Patterning, Temporal Discrimination, and Context‑dependent Decision Making. After training, networks were probed with long sequences of noise or task‑related inputs to characterize their intrinsic dynamics. HOIs were quantified using O‑information and S‑information (KSG estimator, JIDT implementation (1)) across all combinations of 3–8 hidden units. The nonlinear dimensionality of the hidden‑state trajectory was quantified using correlation dimension and complemented by PCA‑based variance analyses.
Results Training induced robust HOIs across tasks, with simpler tasks producing predominantly redundant interactions and more complex tasks eliciting stronger synergistic structure. Crucially, we found a systematic negative correlation between O‑information and the dimensionality of hidden‑state trajectories: networks with more synergy explored higher‑dimensional dynamical manifolds, whereas networks dominated by redundancy collapsed onto lower‑dimensional regimes. This relationship was consistent across tasks, input conditions, and network realizations. Pruning procedures that sparsified the weight matrix did not disrupt the HOI–dimensionality link
Discussion Our results reveal a mechanistic coupling between informational structure and dynamical geometry in recurrent systems: synergy emerges when the network expands its accessible dynamical repertoire, while redundancy reflects a contraction onto lower‑dimensional attractors. This suggests that O‑information can serve as a general marker of dynamical richness and computational flexibility in recurrent architectures. Because the relationship holds across tasks and network configurations, it may reflect a broader organizational principle of recurrent computation. Future work will test whether this coupling persists in multitask settings, under perturbations, and in biologically inspired architectures.
References (1)\tJoseph T. Lizier, "JIDT: An information-theoretic toolkit for studying the dynamics of complex systems", Frontiers in Robotics and AI 1:11, 2014; doi:10.3389/frobt.2014.00011 (pre-print: arXiv:1408.3270)
Acknowledgement This work is funded by Fondecyt grant 1241469 (ANID, Chile). AC3E is funded by Basal grant AFB240002 (ANID, Chile)
