BufNMFCross

Resynthesis of targets using a source's spectral bases

BufNMFCross uses non-negative matrix factorisation (“NMF”, for more info, see BufNMF) to reconstruct the components of a target sound using spectral templates of a source sound. This process is a type of audio mosaiking (or “musaiking”) that is intended to convey certain aspects of a target recording (such as melody and rhythm) using sound components (such as timbre) from a source recording. The result is a hybrid sound whose character depends on how well the target can be represented by the source’s spectral templates.

Target: Drum Loop

Source: Synthesizer sounds

Output:

Rather than replacing single spectral frames in the target with single spectral frames in the source, BufNMFCross looks for opportunities to use a sequence of spectral frames from the source buffer to enhance the perception of the timbral morphology of both the source and target. This functionality can be adjusted using the continuity argument.

Additionally, BufNMFCross will avoid repeating a spectral frame within a specified duration of time (using the argument timeSparsity). This helps avoid a common problem with this type of audio mosaiking: the repetition or overuse of a single sound element from the source. Avoiding recently used sounds is similar to a round-robin functionality in many modern samplers.

Finally, BufNMFCross allows for a spectral frame in the target to be approximated using more than one spectral frame from the source. This can help more closely approximate the spectral frames in the target, however, using too many source spectral frames at once can cause phase cancellation or other undesirable artefacts. This functionality can be adjusted using the polyphony argument, which sets the maximum number of source spectral frames that can be used at one time.

Related Resources