A Malleable Interface for Sonic Exploration

Input devices for controlling music software can benefit fromexploiting the use of perceptual-motor skill in interaction.The project described here is a new musical controller, designed with the aim of enabling intuitive and nuanced interaction through direct physical manipulation of malleablematerial.The controller is made from conductive foam. This foamchanges electrical resistance when deformed; the controllerworks by measuring resistance at multiple points in a single piece of foam in order to track its shape. These measurements are complex and interdependent so an echo statenetwork, a form of recurrent neural network, is employed totranslate the sensor readings into usable control data.A cube shaped controller was built and evaluated in thecontext of the haptic exploration of sound synthesis parameter spaces. Eight participants experimented with the controller and were interviewed about their experiences. Thecontroller achieves its aim of enabling intuitive interaction,but in terms of nuanced interaction, accuracy and repeatability were issues for some participants. It's not clear fromthe short evaluation study whether these issues would improve with practice, a longitudinal study that gives musicians time to practice and find the creative limitations ofthe controller would help to evaluate this fully.The evaluation highlighted interesting issues concerningthe high level nature of malleable control and different approaches to sonic exploration.