Student: Xabier Sarasola
Supervisor: Eva Navas
PhD date of defense: 12/Dec/2020
Abstract:
This thesis focuses on the development of a new bertsolaritza singing voice synthesis system using as base original bertsolaritza live session recordings. The challenge of this work is not only the implementation of a singing voice synthesis system. The recorded corpus of bertsolaritza contains the transcriptions of improvised verses, but the audio files contain multiple elements that are not singing voice. As the majority of the recorded audios are live sessions, the voice of a speaker, applause of the public and noise are part of the database. In addition, the musical labeling of the singing voice is not included in the database. With a database of these properties, the aim of this work is to create methods to clean, segment and label the audios in the bertsolaritza and analyze the possibility of using them to create synthesis models for bertsolaritza singing voice synthesis.
We have developed methods to automatically obtain the singing voice segments in the recordings, creating new speech and singing voice classification algorithms. The segmentation of bertso utterances and phonemes has been performed in a multi-singer database. The segmentation algorithms proposed have the capacity to align material from unseen bertsolaris in the future. After that, we analyzed the musical properties of the bertsolaritza art and compared the theoretical melodies in the database with the actual interpretation of them. We defined automatic systems to musically label the bertsolaritza singing voice generating a fully labeled bertsolaritza database. Musical labeling included vibrato and we analyzed the use of it in each bertsolari. We evaluated all automatic labeling systems in the process.
After creating a labeled database of bertso recordings we generated singing voice synthesis systems using HMM and DNNs. We included f0 normalization, tempo adaptation and vibrato prediction techniques in these systems. We defined methods to automatically adapt music scores for each bertsolari considering the pitch range of each bertsolari. We evaluated synthesis models created for different bertsolaris in a subjective and objective way obtaining good results.
The contributions of this thesis are related to bertsolaritza and singing voice synthesis. We added new information levels to the bertsolaritza corpus with the segmentation of singing voice, the alignment of utterances and phonemes and the subsequent musical labeling. These labeling methods need no manual supervision and therefore we created tools to increase the labeled database in the future. We created a multi-singer singing voice database that is considerably bigger than any state of the art singing voice databases. Finally we defined systems to synthesize bertsolaritza singing voice using different singers and technologies obtaining positive results.
