A Pattern Theory approach to Generative Media
The project presents an adaptation of machine cognitive models for the creative and interactive design of media, including sound. The result may be variously understood as:
- A tool for statistical performance
- A system for eliciting the self organization of structures in time and parameter space
- A model for the spontaneous organization of real-time series
- A hidden variables stochastic graphical model for the succession and time of onset of sensory events or changes in sensory parameters
- A repurposing of the hidden Markov model toward capturing and synthesizing patterns with perceptually natural qualities
interactive, statistical composition
The system in practice acts like a statistical score - a network (directed graph) of probabilistic "notes", having statistical numerical properties and duration.
An excitation source gives rise to note-chains, one for each excitation, based on the propagation of tokens which are spawned within the network.
The resulting output consists of a time-series of parameter vectors. Each vector should be thought of as descriptive of the given fragment of musical time, and one is elicited by each move of a token. The shape of the set of parameter curves which are generated recapitulate the network statistics.
These statistics are shaped in real-time to spontaneously form patterns along the duration of the trajectories of each token.
self (dis)organization: order, disorder, and transitions
A further theme in the present work concerns performance or interactivity in the context of indeterminacy. A basic function of this system is to facilitate interactive control over the spontaneous formation of structure.
An adaptation scheme is adopted which allows for the refinement of model parameters in a way that is intended to form interesting (e.g. musical) structures in its output, eliciting features like repetition, variance, and transience. This is accomplished by adapting the model parameters at or near a state which has produced a given output to the output parameters themselves.
Independent influence over time and parameter-space structures in the resulting patterns may be effected, by means of a set of real-time sequencing controls which are accessible to the user.
Consequently, if the system has localized reasonably to a pattern, the latter may be deconstructed and reconstructed with or without affecting the underlying model data. For example, a transition probability floor may be raised or lowered to perturb temporal structures in a given familty of sequences (figure right).
software for interactive pattern synthesis
A general reference implementation has been developed, together with an interface layer which uses the Max/MSP SDK (Cycling'74), exposing a variety of controls over its structure and evolution.
The software allows to experiment or design with the spontaneous organization of media (sound, video, image, ..) and may be freely downloaded.
- Software (interactive HMM for Max-MSP, compiled for Mac OS X)