Quantifying Musical Micro-Motion: Applying Optical Flow Analysis to Audiovisual Abstract Animation

Abstract

Optical flow analysis, while initially developed for computer vision, has expanded its applications into various domains. While traditional cognitive models of musical expression focus on understanding the impact of music on listeners, they often show less interest in what the music itself expresses or fail to fully elucidate the complexities associated with this sphere. Moreover, these models tend to be coarse and predominantly top-down, categorising music based on standard emotion theories using general musical dimensions, without considering the intricacies of perceptual abstraction and idiosyncratic processing. In contrast, contemporary computer software allows for a more nuanced exploration of music aesthetics by integrating music visualisation into optical flow analysis. This article explores the potential of this integration, extending the application of algorithmic music visualisation to serve as a tool for feature detection and audiovisual priming at a finer scale. Optical flow analysis facilitates the collection and evaluation of data that transcends intuitive audiovisual experience, offering a method which may lead to a more “objective” understanding of aesthetic dimensions. By analysing micro-motion and micro-expression in abstract animations of the acoustic spectrum, this approach opens avenues to subtleties previously unexplored and deemed “ineffable”. Through two case studies involving Aphex Twin’s “Bucephalus Bouncing Ball” and Frédéric Chopin’s Prelude op. 28 no. 3, this article demonstrates the potential of optical flow analysis in uncovering hidden layers of meaning in musical expression via idiosyncratic, animated representations of spectral data.