Designing digital interfaces for enhanced musical interaction and digital instrument mapping strategies to improvise alongside acoustic instruments

Abstract

Today, with the increased accessibility of powerful processors and the development of sophisticated musical software, digital musical instruments, and interactive music systems have become quite common. It is now possible for sound artists to develop their customized musical instruments or interactive music systems to realize musical ideas that they have in their minds. Various commercialized digital control interfaces like Leap Motion Tracker or Playtronica's Playtron, which developed in the last two decades, offered great potential in musical applications. The advancements in human-computer interaction fashion led to much more expressive and engaging designs. This thesis is written from a unified perspective of a designer/composer/performer and proposes two newly developed interactive music systems: 'Lemohaxis' and 'Guru'. This study examines Lemohaxis and Guru as the main outputs of the study with focusing on their design processes and discusses their place in the literature. Additionally, in order to understand the effects of design decisions on the systems, these two systems are compared in detail. A limited number of performances were evaluated through various data collecting methods including questionnaires and unstructured interviews with the performers and the audience. The results of this research show that gestural controls with a Leap motion tracker can be much more expressive when compared to an Xbox controller. Also, simple one-to-one mappings for gestural controls allow a high level of performability for novice users without lowering the expressivity as long as it is possible to perform theatrics intuitively, yet it limits the complexity of musical controls. Comparison between two example systems show that there is a tradeoff between sonic expressivity and performability. In addition, the unique perspective in exemplary design of 'Lemohaxis', emphasizes that it is possible to achieve the targeted learning curve with unconventional approaches. Another innovative approach that is suggested by 'Guru' system is, using objects that are related to sonic outcome for switching between mapping presets of the leap motion to let the audience to relate the performers actions and the resulting sound for improving transparency. The concepts that are vital to an interactive music system design are introduced through a review of the literature. Similar systems and digital instruments are examined in terms of aesthetics and structural design. Structures of systems, mapping strategies, aesthetic needs, and decision-making on control interfaces are discussed in detail. Structural design, musical output, and the artistic needs that the systems must meet, are compared with similar digital instruments and interactive systems in the field.