Introduction Convolutional networks are commonly used in brain models [e.g. 3, 5]. Convolutions assume a uniform connection structure from source to target. However, a recent analysis of mesoscale connectivity in the mouse cortex [4] showed that many connections over and under-represent parts of the source and/or target area, and may project distant parts of a source into nearby parts of a target. Because multiple connections converge in each cortical area, these complex projection structures constrain which signals converge in the brain. Here we analyze the structural error inherent in using standard convolutional networks to model the mouse cortex, and develop an extension that has a more accurate structure.
Methods We first calculated 2D cortical-surface coordinates for each voxel of each mouse neocortical area, as in [4], and normalized them to have standard deviations (SD) of 1. For each projection, we used voxel-voxel connection density estimates from [2] to calculate the weighted-mean source-area coordinates that provided input to each target-area voxel. We evaluated potential remapping functions (f) that mapped target coordinates (t) to corresponding source coordinates (s), such that s = f(t). We first fitted a hierarchy of constrained geometric models, including rotation/flip, linear, and affine remapping. We also introduced a more flexible nonlinear remapping model based on a radial basis function interpolator with thin-plate spline kernel [1].
Results We calculated the mean Euclidean distance dμ between actual source coordinates and those predicted by each function. Rotation/flip remapping, which is consistent with standard convolution (accounting for different map orientations in cortical coordinates), gave the poorest fit (dμ = 0.62 over all connections, vs. coordinate SD=1). Linear and affine remapping, which correspond to convolution with non-integer stride and offset, reduced the mean error to 0.17 and 0.10, respectively. However, large errors remained in some voxels for each of these models. The general RBF model further reduced mean error to < 0.001, with no large errors. This model corresponds to convolution layers followed by general data-driven remapping layers.
Discussion This analysis shows that standard convolutional networks are inconsistent with the mesoscale connectivity of mouse cortex. However, realistic connectivity can be recovered by inserting remapping layers into convolutional networks. This proposed extension improves approximation of a wide range of complex cortical structures, including diverse subfields in higher visual areas, low-level multisensory connections, and complex convergence patterns in prefrontal areas.
References [1] GE Fasshauer. “Meshfree Approximation Methods with Matlab, World Sci”. In: Publishing Co, Singapore (2007). [2] Joseph E Knox et al. “High-resolution data-driven model of the mouse connectome”. In: Network Neuroscience 3.1 (2018), pp. 217–236. [3] Jonathan A Michaels et al. “A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping”. In: Proceedings of the national academy of sciences 117.50 (2020), pp. 32124–32135. [4] Kinjal Patel et al. “Spatial organization of multisensory convergence in mouse isocortex”. In: bioRxiv (2024), pp. 2024–12. [5] Jianghong Shi et al. “MouseNet: A biologically constrained convolutional neural network model for the mouse visual cortex”. In: PLOS Computational Biology 18.9 (2022), e1010427.
Acknowledgement This work was supported by NSERC Discovery Grant RGPIN-2025-04919.
