Multi Modal Instrument is a platform I’m developing for controller evaluation in the context of Physical Modelling synthesis, with a focus on affordable controllers. MMI provides a main editor for configuring the synthesis engine, and a number of editors for specifying the mappings of controller sensors onto synthesis parameters. MMI also supports the use of OSC messages from any source, and so controllers which do not yet have a MMI patch can also be used, and software such as Reactivision (fiducial marker tracking).
Feedback to the musician is in the form of the audio from the synthesis, a 3D animation of the physical model and force feedback is provided for some haptic controllers (Novint Falcon, Logitech Force3D Pro).
Current work is focussing on developing analytical software providing evaluation metrics based on assessment against audio targets.
MMI is written in Max/MSP/Jitter and uses a number of external objects for synthesis and control:
IRCAM’s Modalys (physical modelling synthesis).
aka.wiiremote (wii remote + nunchuck)
cv.jit, and jit.freenect.grab (blob tracking and Kinect)
fingerpinger (multitouch trackpad)
np_nifalcon (Novint Falcon)
kinect_skeleton (an experimental port of OSCeleton to a Max object)
‘Multi-Modal Instrument: Towards A Platform for Comparative Controller Evaluation’ (Polfreman, R), In Proceedings of the 2011 International Computer Conference (Huddersfield: ICMA, 2011).