MIXPREP: Machine learning-based multitrack mix preparation assistant

Abstract

Music production is a general term for describing a set of complicated processes where artistic and technical efforts are involved. Besides the artistic part, the technical side of some parts has regular iterative works. This study focuses on the mix preparation step of the multitrack audio mixing stage in music production by seeking an automatic software solution regarding the intelligent music production paradigm. The structure of the dissertation consists of four components: Theoretical background with fundamental definitions of knowledge both in music production analysis, instrument recognition theories and applications, the approach and explanation of the development of the proposed assistant software, and last but not least, an experiment stage comprising of performance testing with many multitrack projects. Before diving into the development stage, the perspective and the definition of the mix preparation are presented after introducing the music production with a brief historical background. Afterwards, delineation of the intelligent music production research field apart from subjective interests takes part. Instrument recognition literature takes an important part in the conceptualization of the automatic mix preparation solution. Because of that, an extensive historical background in the instrument recognition field is given without getting into redundant theoretical aspects. Apart from that, a reasonable amount of information about the definition of the fundamental concepts of digital audio, audio content analysis and machine learning seemed appropriate to be mentioned since the audience of this research addressed the music technology field. After providing the fundamental theoretical background, the software development approach for the mix preparation assistant is presented. This section explains the software structure by stating the basic requirements of the mix preparation regarding design concerns of the graphical user interface (GUI) consideration for practical usage. The main issues are the GUI layout, software usage, and building a dataset with a related machine learning model. Eventually, a loop-based audio dataset creation approach and ML model are put forward by testing their performance with many audio files from 80 multitrack audio projects in four musical genres (Pop, Rock, Jazz, Electronic/Dance). The experiment is set concerning instrument families provided in the dataset and genre-related performance estimations of each one. The results were interpreted by accentuating the crucial points of implementing the ML-based mix preparation solution. Detailed evaluation results are in the appendices. This study proposes a concept of intelligent mix preparation software by providing a methodology for the design concept and application.