Introduction
Visual information from individual photoreceptors represents only simple light intensity information. More complex visual information, such as motion, is discerned by considering the combined responses from several receptors, or over a duration. The exact processes and locations at which encoding steps occur along the visual pathway are unclear. Yet, by aligning response preferences of neurons to the presence of specific visual stimuli, specialised encoding regions may be identified. Using computer vision methods, we demonstrate the ability to extract simple visual components from natural stimuli and, using electrophysiological data in mice, predict neuronal optical flow response preferences across the visual pathway.
Methods
Electrophysiological recordings from the public Allen Brain Observatory dataset, comprising responses of 32 mice (Mus musculus) to varied artificial and natural stimuli, were processed to detect spiking action potentials [1]. Dense optical flow analysis was performed to extract motion magnitude and direction by estimating local neighbourhood displacement between frames. Magnitudes were weighted by their cosine direction components to assess correlation between spiking rate and motion magnitude in 8 directions. Moreover, region-specific logistic regression models were trained, using either drifting grating or natural video stimulus-response data, to predict predominant global motion direction for a novel natural video from spiking rates.
Results
Subsets of neurons within regions displayed correlation between spiking rate and optical flow magnitude consistently across repeated presentations, but due to motion direction bias within the video, horizontal direction preferences were more represented than vertical ones. Regional regression models were able to predict predominant motion direction, with accuracy varying across regions, and specific direction performance reliant on sufficient training examples. Both models trained using only drifting gratings, or only natural video, displayed high direction prediction accuracy to a novel video. Hence, we identify a subset of visual pathway cells with directional coding preferences to natural video motion consistent with rate-based coding.
Discussion
This study was motivated by previous attempts to train models to predict high-resolution pixel images from spiking activity, whereby models were unable to generalise to predict novel stimuli [2]. Our work elucidates possible shortcomings in such an approach that warrant further investigation: mouse spiking activity contains less relevant pixel information than we previously believed, quantified by our analysis, likely due to a specialisation for motion over acuity. This study represents the first to compare regional differences in visual feature prediction, using electrophysiological activity, for a novel natural video. Future work aims to explore more specific feature predictions, including foreground/background motion discrimination.
References
1. Siegle, J. H., Jia, X., Durand, S., Gale, S., Bennett, C., Graddis, N., Heller, G., Ramirez, T. K., Choi, H., Luviano, J. A., Groblewski, P. A., Ahmed, R., Arkhipov, A., Bernard, A., Billeh, Y. N., Brown, D., Caldejon, S., Casal, L., Cho, A., … Koch, C. (2021). Survey of spiking in the mouse visual system reveals functional hierarchy. Nature, 592(7852), 86–92. https://doi.org/10.1038/s41586-020-03171-x
2. Chen, Y., Beech, P., Yin, Z., Jia, S., Zhang, J., Yu, Z., & Liu, J. K. (2024). Decoding dynamic visual scenes across the brain hierarchy. PLoS Computational Biology, 20(8), e1012297. https://doi.org/10.1371/journal.pcbi.1012297
Acknowledgement
This project utilises the open source Allen brain observatory visual coding neuropixels dataset from the Allen Institute for Brain Science [1].
