DeepRESTORE aims to investigate the use of silent speech interfaces to restore communication in people who have been deprived of the ability to speak. Silent speech interfaces are devices that capture non-acoustic biological signals generated during the speech production process and use them to predict the intended message. While SSI interfaces have been primarily investigated in the context of silent speech-to-text ( Speech – to-Text) recognition, this project will also investigate direct speech synthesis techniques, thereby directly generating the corresponding speech waveform for their output.

There are two main biological signals that will be investigated in this project. In subproject 1, electromyographic signals produced by the speech production apparatus will be captured and processed along with videos of facial movements. Subproject 2 will address the very novel field of brain-computer interfaces using brain signals obtained through electroencephalography (both intracranial and noninvasive). From the entire spectrum of speech disorders that can affect a person’s voice, DeepRESTORE will address two conditions, each being the focus of a particular subproject :

• laryngectomy patients (SP1), whose upper vocal tract (i.e. mouth, tongue, lips, etc.) is intact, but they are unable to speak normally because their entire larynx (an essential organ for sound production) has been surgically removed as a treatment for laryngeal cancer. These individuals retain control over the speech articulators and therefore silent speech data reflecting articulator movements can be captured using EMG for example.
• Neurodegenerative diseases and other traumatic injuries (SP2) can leave an individual paralyzed and possibly unable to speak. For many of these individuals, their only means of communication is through limited eye movements and blinking; however, for those with full locked-in syndrome, even this type of communication may not be possible. A SSI-based communication system could provide them with a more effective and efficient way to communicate without the need for a caregiver to initiate a conversation or speak for them. Such technology could therefore dramatically improve the lives of these individuals.

DeepRESTORE will be carried out by two research groups located in Spain with a strong collaboration of external researchers. This project represents the continuation of the collaboration developed within the framework of the ReSSInt coordinated project , dedicated to silent voice research. The Aholab group of the UPV/EHU has demonstrated its experience in the field of signal processing and voice synthesis. The UGR group is led by a researcher with consolidated research experience in the field of silent voice interfaces. Both groups will be supported by researchers specialised in the area at an international level.