CNS*2026 Halifax

Introduction
Motion perception is the remarkable ability of the visual system to recognize complex human movements effortlessly. Computational movement analysis seeks representations that mirror this efficiency while remaining interpretable. The motor modularity hypothesis proposes that movements are composed of weighted primitives [1], yet whether theory-driven decompositions outperform alternative approaches remains untested. We compared Temporal Movement Primitives (TMPs), Legendre polynomial coefficients, and autoencoder embeddings to ask which movement features enable motion classification and assess if the chosen movement features align with how human observers discriminate actions, paralleling perceptual research on form versus motion cues [2].


Methods
We analyzed videos of 16 daily activities from the MoVi dataset [3] and extracted joint-angle trajectories using MMPose, with segmentation via visual inspection. We represented these trajectories in three ways: 1- TMP weights from Bayesian decomposition with Gaussian process priors for varying numbers of primitives (1- 20); 2- Legendre polynomial coefficients for varying maximum degrees (0–10); 3- latent vectors from an encoder-decoder network. To determine which features drive discrimination, we assessed cross-validated classification accuracy, optimal primitive count and polynomial degree, reconstruction quality, and interpretability. To isolate dynamics from posture, we repeated analyses after subtracting mean joint positions per trial.


Results
Legendre coefficients achieved 96% cross-validated classification accuracy across 16 classes, outperforming TMP weights (91%) and autoencoder features (85%). Optimal TMP count was 5 primitives, and the optimal Legendre degree was 0, revealing postural configuration, not temporal dynamics, is the primary discriminative feature. After posture removal, degree-2 polynomials captured remaining discriminative dynamics. Classification and movement generation dissociated: when generating movements from averaged category weights, TMPs preserved dynamics, producing natural motion, whereas Legendre-generated movements retained original posture but with unclear motion. L1 regularization identified 10 joints carrying the most discriminative information.


Discussion
Posture dominance for activity classification aligns with biological motion perception, showing form cues often suffice for action-type recognition. The dissociation between classification and generation shows discriminative and generative adequacy are distinct properties: Legendre coefficients excel at categorization, TMPs preserve temporal structure for synthesis, and autoencoders achieve optimal dimensionality reduction from 240 (5 primitives × 48 coordinates) to 32. Beyond classification, these results reveal that movement is organized into separable postural and dynamic components, opening avenues to explore minimum temporal duration for motion perception, whether partial cycles suffice, and how accuracy scales with available dynamics.


References

1. Knopp, B., Velychko, D., Dreibrodt, J., & Endres, D. (2019b). Predicting perceived naturalness of human animations based on generative movement primitive models. ACM Transactions on Applied Perception, 16(3), 1–18. https://doi.org/10.1145/3355401
2. Lange, J., & Lappe, M. (2006). A Model of Biological Motion Perception from Configural Form Cues. Journal of Neuroscience, 26(11), 2894–2906. https://doi.org/10.1523/jneurosci.4915-05.2006
3. Ghorbani, S., Mahdaviani, K., Thaler, A., Kording, K., Cook, D. J., Blohm, G., & Troje, N. F. (2021). MoVi: A large multi-purpose human motion and video dataset. PLoS ONE, 16(6), e0253157. https://doi.org/10.1371/journal.pone.0253157 
   
   

Acknowledgement
This work was supported by the Natural Sciences and Engineering Research Council (NSERC) of Canada. The research was undertaken thanks in part to funding from the Connected Minds Program, supported by Canada First Research Excellence Fund, Grant #CFREF-2022-00010. Also, we thank the creators of the MoVi dataset for making their data publicly available.