LSTM-Based Framework for the Synthesis of Original Soundtracks

Abstract

Recently, significant developments have been made in Long Short-Term Memory (LSTM) networks within the realm of synthesis music. Notwithstanding these advancements, several challenges persist, warranting further research. Primarily, there exists an absence of dedicated research on the application of LSTM networks for the synthesis of Original Sound Tracks (OST). Secondly, in general, we can only judge whether the synthesized music meets our expectations based on the model output however, due to the time-consuming of training the model may need to try multiple times to obtain successful training results. Moreover, the subjective of music quality evaluation relying on human perception, not only the result of model training. To address these multifaceted challenges, this paper concentrates specifically on OST and proposes a framework termed the OST Synthesis Framework (OSTSF) utilizing LSTM. This framework accepts various OST types as input, processed through LSTM to yield innovative OST. Additionally, a novel preprocessing algorithm is proposed to screen input OST elements such as notes and chords, enabling control over music type and quality before the training phase. This algorithm serves to mitigate training uncertainties and reduce situations that require repeated training. Besides, a postprocessing approach, leveraging mathematical formulations facilitates the evaluation of synthesis OST also proposed. This approach aims to quantify subjective evaluations, providing a more intuitive representation through scoring metrics. Experiment results reveal that the OSTSF synthesized OST received favorable rate among a cohort of 100 surveyed respondents attaining 78.8%, demonstrating the efficacy of the proposed framework in the realm of music synthesis utilizing LSTM.