Multimodal Music and Lyrics Fusion Classifier for Artist Identification

Aryafar, Kamelia; Shokoufandeh, Ali

doi:10.1109/icmla.2014.88

Cited by 6 publications

(5 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This group thinks of multimodality as different communication methods: visual, linguistic, spatial, aural, and gestural (11). Multimodal music datasets within this context may incorporate audio, gestures, and written documents, such as lyrics (12).…”

Section: Existing Definitionsmentioning

confidence: 99%

Multimodal Music Datasets? Challenges and Future Goals in Music Processing

Christodoulou,

Lartillot,

Jensenius

2024

Preprint

View full text Add to dashboard Cite

The term ``multimodal music dataset'' is often used to describe music-related data that represent music as a multimedia art form and multimodal experience. However, the term ``multimodality'' may mean different things in related disciplines, such as musicology, music psychology, and music technology. This paper proposes a definition of multimodality that works across different music disciplines. Many challenges are related to constructing, evaluating, and using multimodal music datasets. We provide a task-based categorization of these datasets and explore theoretical methodologies aimed at enhancing their future construction. Diverse data preprocessing methods are illuminated, highlighting their contributions to transparent music analysis. Additionally, evaluation metrics, methods, and benchmarks tailored for multimodal music processing tasks are scrutinized, empowering researchers to make informed decisions and facilitating cross-study comparisons.

show abstract

Section: Existing Definitionsmentioning

confidence: 99%

Multimodal Music Datasets? Challenges and Future Goals in Music Processing

Christodoulou,

Lartillot,

Jensenius

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The audio domain has 3 articles, where recognition [391], detection [392] and translation [393] each have 1. Detection [394], prediction [395] and translation [396] each have one paper from 3 papers in the game domain.…”

Section: Inclusion Criteriamentioning

confidence: 99%

“…[98], [100], [104], [116], [125], [127], [187], [206], [241], [326], [365], [395], [411], Image & Numerical [62], [75], [119], [126], [167], [313], [331], [353], [405], [410], Audio & Text & Sensor [384], Audio & Text [180], [282], [377], [391], [392], Text & Signal [109], Text & Numerical [304], [349], Sensor & Signal [240], [242], [258], [389], Sensor & Numerical [183], Signal & Numerical [205], [257], [260], [318]. Figure 10 displays the extracted information related to each modality and data type with the links between them.…”

Section: B Taskmentioning

confidence: 99%

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

2023

View full text Add to dashboard Cite

Multimodal machine learning (MML) is a tempting multidisciplinary research area where heterogeneous data from multiple modalities and machine learning (ML) are combined to solve critical problems. Usually, research works use data from a single modality, such as images, audio, text, and signals. However, real-world issues have become critical now, and handling them using multiple modalities of data instead of a single modality can significantly impact finding solutions. ML algorithms play an essential role by tuning parameters in developing MML models. This paper reviews recent advancements in the challenges of MML, namely: representation, translation, alignment, fusion and co-learning, and presents the gaps and challenges. A systematic literature review (SLR) applied to define the progress and trends on those challenges in the MML domain. In total, 1032 articles were examined in this review to extract features like source, domain, application, modality, etc. This research article will help researchers understand the constant state of MML and navigate the selection of future research directions.

show abstract

“…• artist identification, through lyrics and audio fusion [38]; • derivative works classification of youtube video through audio, video, titles and authors [39]; • instrument classification by exploiting audio recordings and performance video [40], [41]; • tonic identification, that is: given an audio recording and the note level, find the tonic [42]; • expressive musical description, which consists in associating a musical annotation to an audio recording by extracting features with the help of symbolic level [43].…”

Section: Classificationmentioning

confidence: 99%

“…conversion from one modality to the other, such as in query-by-humming -which includes a conversion from audio to the symbolic level -or in audioto-score alignment where symbolic scores can be converted to audio through a synthesis process. feature selection through Linear Discriminant Analysis (LDA) [28] or ReliefF [43] normalization of the extracted features [48] source-separation in lyrics-to-audio alignment and source association [63], [64] chord labeling on audio only [62] multi-pitch estimation on audio only [65] video-based hand tracking [59] tf-idf -based statistics -see section V-C -adapted for audio [38] Finally, we think that a step worthy of a particular attention is the conversion to a common space of the extracted features, to make them comparable. We will talk about this step in section VI.…”

Section: Data Pre-processingmentioning

confidence: 99%

Multimodal Music Information Processing and Retrieval: Survey and Future Challenges

Simonetta

Ntalampiras

Avanzini

2019

2019 International Workshop on Multilayer Music Representation and Processing (MMRP)

View full text Add to dashboard Cite

Towards improving the performance in various music information processing tasks, recent studies exploit different modalities able to capture diverse aspects of music. Such modalities include audio recordings, symbolic music scores, midlevel representations, motion and gestural data, video recordings, editorial or cultural tags, lyrics and album cover arts. This paper critically reviews the various approaches adopted in Music Information Processing and Retrieval, and highlights how multimodal algorithms can help Music Computing applications. First, we categorize the related literature based on the application they address. Subsequently, we analyze existing information fusion approaches, and we conclude with the set of challenges that Music Information Retrieval and Sound and Music Computing research communities should focus in the next years.

show abstract

Multimodal Music and Lyrics Fusion Classifier for Artist Identification

Cited by 6 publications

References 12 publications

Multimodal Music Datasets? Challenges and Future Goals in Music Processing

Multimodal Music Datasets? Challenges and Future Goals in Music Processing

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

Multimodal Music Information Processing and Retrieval: Survey and Future Challenges

Contact Info

Product

Resources

About