mmi

mmi showing a circular plate being struck

Multi Modal Instrument is a platform I’m developing for controller evaluation in the context of Physical Modelling synthesis, with a focus on affordable controllers. MMI provides a main editor for configuring the synthesis engine, and a number of editors for specifying the mappings of controller sensors onto synthesis parameters. MMI also supports the use of OSC messages from any source, and so controllers which do not yet have a MMI patch can also be used, and software such as Reactivision (fiducial marker tracking).

Synthesis animation

An early Kinect hand tracking interface

Feedback to the musician is in the form of the audio from the synthesis, a 3D animation of the physical model and force feedback is provided for some haptic controllers (Novint Falcon, Logitech Force3D Pro).

Current work is focussing on developing analytical software providing evaluation metrics based on assessment against audio targets.

MMI is written in Max/MSP/Jitter and uses a number of external objects for synthesis and control:

IRCAM’s Modalys (physical modelling synthesis).

aka.wiiremote (wii remote + nunchuck)

cv.jit, and jit.freenect.grab (blob tracking and Kinect)

fingerpinger (multitouch trackpad)

np_nifalcon (Novint Falcon)

kinect_skeleton (an experimental port of OSCeleton to a Max object)

Reactivision can be used to control the synthesis

A Novint Falcon provides a haptic interface

An experimental Kinect interface using a skeleton model

Related Publications:

‘Multi-Modal Instrument: Towards A Platform for Comparative Controller Evaluation’ (Polfreman, R), In Proceedings of the 2011 International Computer Conference (Huddersfield: ICMA, 2011).