[et_pb_section fb_built=”1″ _builder_version=”4.27.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_row _builder_version=”4.27.0″ _module_preset=”default” max_width=”80%” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.27.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_blurb title=”Subproject 1″ image=”https:\/\/aholab.ehu.eus\/deeprestore\/wp-content\/uploads\/2024\/07\/DEEP-RESTORE-favicon.webp” icon_placement=”left” content_max_width=”100%” _builder_version=”4.27.2″ _module_preset=”default” header_text_color=”gcid-primary-color” header_font_size=”20px” body_text_color=”gcid-secondary-color” body_font_size=”25px” custom_margin=”||||false|false” custom_padding=”||7px||false|false” animation=”off” border_width_bottom=”1px” border_color_bottom=”gcid-primary-color” global_colors_info=”{%22gcid-primary-color%22:%91%22border_color_bottom%22,%22header_text_color%22%93,%22gcid-secondary-color%22:%91%22body_text_color%22%93}”]Deep learning for speech restoration from facial movement biosignals[\/et_pb_blurb][\/et_pb_column][\/et_pb_row][et_pb_row column_structure=”3_5,2_5″ make_equal=”on” _builder_version=”4.27.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_column type=”3_5″ _builder_version=”4.27.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.27.2″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n
DeepRESTORE aims to investigate the use of Silent Speech Interfaces (SSI) to restore communication in people who have been deprived of the ability to speak. Silent Speech Interfaces are devices that capture non-acoustic biological signals generated during the speech production process and use them to predict the delivered message. While SSIs have been primarily investigated in the context of speech recognition ( Silent – Speech -to-Text), this project will also investigate direct speech synthesis techniques, thereby directly generating the corresponding speech waveform.<\/p>\n
Of the two biological signals to be investigated in the coordinated project, subproject 1 will capture two biosignals produced by movements of the speech-producing apparatus: electrical signals generated by the muscles of the face and neck involved in speech production ( sEMG signals ) and captured video images of the face. Using a set of sensors located on the face and throat, and a camera, and without the use of acoustic signals, the SSI device will decode these signals into the corresponding text (EMG-to-text) or acoustic (EMG-to-speech) message. For this purpose, algorithmic techniques based on deep learning will be used.<\/p>\n
Although silent speech interfaces can be used in other contexts (such as in a security context, to maintain privacy on a telephone), our project focuses on providing speech to people who have undergone total laryngectomy surgery . After a period of intensive learning, these people typically regain a so-called esophageal voice, the characteristics of which are markedly different from healthy speech. Since they still retain control over the speech articulators, silent speech data reflecting the articulator movements can be captured and converted into artificial speech.<\/p>\n
sEMG- based SSI devices can significantly improve the quality of life of these people. During the project, the existing databases in Spanish will be completed with more data in other languages (English) that will be available to the research community. In addition, the use of SoA Deep Neural Networks will be further explored, contributing to new learning architectures. The project will be carried out in collaboration with international experts in the field of silent speech, and will collaborate with the Association of Laryngectomees of Bizkaia not only for the collection of data, but also and more importantly for the evaluation and validation of the techniques developed.[\/et_pb_text][et_pb_blurb title=”Goals” image=”https:\/\/aholab.ehu.eus\/deeprestore\/wp-content\/uploads\/2024\/07\/DEEP-RESTORE-favicon.webp” icon_placement=”left” content_max_width=”100%” _builder_version=”4.27.2″ _module_preset=”default” header_font_size=”23px” animation=”off” border_width_bottom=”1px” border_color_bottom=”gcid-primary-color” global_colors_info=”{%22gcid-primary-color%22:%91%22border_color_bottom%22%93}”][\/et_pb_blurb][et_pb_blurb title=”Multi -language database for multilingual video-based EMG and SSI studies for English and Spanish” use_icon=”on” font_icon=”||fa||900″ icon_placement=”left” image_icon_width=”25px” content_max_width=”100%” _builder_version=”4.27.2″ _module_preset=”default” header_font_size=”17px” header_line_height=”1.2em” width=”100%” custom_margin=”||4px||false|false” custom_css_blurb_title=”margin-left: -20px;” global_colors_info=”{}”][\/et_pb_blurb][et_pb_blurb title=”EMG+video- to-text system for Spanish” use_icon=”on” font_icon=”||fa||900″ icon_placement=”left” image_icon_width=”25px” content_max_width=”100%” _builder_version=”4.27.2″ _module_preset=”default” header_font_size=”17px” header_line_height=”1.2em” width=”100%” custom_margin=”||4px||false|false” custom_css_blurb_title=”margin-left: -20px;” global_colors_info=”{}”][\/et_pb_blurb][et_pb_blurb title=”Video+EMG- based speech generation system for Spanish” use_icon=”on” font_icon=”||fa||900″ icon_placement=”left” image_icon_width=”25px” content_max_width=”100%” _builder_version=”4.27.2″ _module_preset=”default” header_font_size=”17px” header_line_height=”1.2em” width=”100%” custom_margin=”||4px||false|false” custom_css_blurb_title=”margin-left: -20px;” global_colors_info=”{}”][\/et_pb_blurb][et_pb_blurb title=”Develop novel techniques for cross-language adaptation and learning transfer in EMG+video- based speech generation systems .” use_icon=”on” font_icon=”||fa||900″ icon_placement=”left” image_icon_width=”25px” content_max_width=”100%” _builder_version=”4.27.2″ _module_preset=”default” header_font_size=”17px” header_line_height=”1.2em” width=”100%” custom_margin=”||4px||false|false” custom_css_blurb_title=”margin-left: -20px;” global_colors_info=”{}”][\/et_pb_blurb][\/et_pb_column][et_pb_column type=”2_5″ _builder_version=”4.27.0″ _module_preset=”default” custom_padding=”|||10px|false|false” border_width_left=”1px” border_color_left=”#dbdbdb” global_colors_info=”{}”][et_pb_image src=”https:\/\/aholab.ehu.eus\/deeprestore\/wp-content\/uploads\/2024\/09\/N004.webp” title_text=”N004″ _builder_version=”4.27.0″ _module_preset=”default” global_colors_info=”{}”][\/et_pb_image][ba_dual_button btn_alignment=”center” btn_a_text=”Subproject 1 (SP1)” btn_a_link=”@ET-DC@eyJkeW5hbWljIjp0cnVlLCJjb250ZW50IjoicG9zdF9saW5rX3VybF9wYWdlIiwic2V0dGluZ3MiOnsicG9zdF9pZCI6IjIwNiJ9fQ==@” btn_b_text=”Subproject 2 (SP2)” btn_b_link=”@ET-DC@eyJkeW5hbWljIjp0cnVlLCJjb250ZW50IjoicG9zdF9saW5rX3VybF9wYWdlIiwic2V0dGluZ3MiOnsicG9zdF9pZCI6IjIwNyJ9fQ==@” connector_type=”icon” connector_icon=”$||divi||400″ button_gap=”10px” _builder_version=”4.27.2″ _dynamic_attributes=”btn_a_link,btn_b_link” _module_preset=”default” custom_btn_a=”on” btn_a_text_size=”1rem” btn_a_text_color=”#FFFFFF” btn_a_bg_color=”#209fc2″ btn_a_border_width=”1px” btn_a_border_color=”#FFFFFF” btn_a_border_radius=”9px” btn_a_icon=”$||divi||400″ custom_btn_b=”on” btn_b_text_size=”1rem” btn_b_text_color=”#FFFFFF” btn_b_bg_color=”gcid-secondary-color” btn_b_border_width=”1px” btn_b_border_color=”#FFFFFF” btn_b_border_radius=”12px” btn_b_use_icon=”off” hover_enabled=”0″ global_colors_info=”{%22gcid-secondary-color%22:%91%22btn_b_bg_color%22%93}” sticky_enabled=”0″][\/ba_dual_button][et_pb_blurb title=”Latest publications” image=”https:\/\/aholab.ehu.eus\/deeprestore\/wp-content\/uploads\/2024\/07\/DEEP-RESTORE-favicon.webp” icon_placement=”left” content_max_width=”100%” _builder_version=”4.27.2″ _module_preset=”default” header_text_color=”gcid-secondary-color” body_text_color=”gcid-secondary-color” width=”100%” animation=”off” hover_enabled=”0″ custom_css_blurb_title=”margin-top: 5px;||margin-left: -10px;” border_width_bottom=”1px” border_color_bottom=”gcid-primary-color” global_colors_info=”{%22gcid-secondary-color%22:%91%22body_text_color%22,%22header_text_color%22%93,%22gcid-primary-color%22:%91%22border_color_bottom%22%93}” sticky_enabled=”0″][\/et_pb_blurb][ba_post_list posts_number=”4″ excerpt_length=”100″ item_spacing=”30px” item_padding=”0px|0px|30px|0px|false|false” image_spacing=”23px” meta_spacing=”5px” excerpt_spacing=”10px” _builder_version=”4.27.0″ _module_preset=”default” title_font=”|700|||||||” title_text_color=”gcid-secondary-color” title_font_size=”1.2rem” content_text_color=”#354559″ content_font_size=”1rem” meta_font=”|700|||||||” meta_text_color=”#354559″ meta_font_size=”1rem” border_radii_image=”on|3px|3px|3px|3px” border_width_bottom_post=”1px” border_color_bottom_post=”#e2e5ed” global_colors_info=”{%22gcid-secondary-color%22:%91%22title_text_color%22%93}”][\/ba_post_list][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":"
Deep learning for speech restoration from facial movement biosignalsSP1 Description DeepRESTORE aims to investigate the use of Silent Speech Interfaces (SSI) to restore communication in people who have been deprived of the ability to speak. Silent Speech Interfaces are devices that capture non-acoustic biological signals generated during the speech production process and use them to […]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"class_list":["post-435","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/aholab.ehu.eus\/deeprestore\/en\/wp-json\/wp\/v2\/pages\/435","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/aholab.ehu.eus\/deeprestore\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/aholab.ehu.eus\/deeprestore\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/aholab.ehu.eus\/deeprestore\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/aholab.ehu.eus\/deeprestore\/en\/wp-json\/wp\/v2\/comments?post=435"}],"version-history":[{"count":3,"href":"https:\/\/aholab.ehu.eus\/deeprestore\/en\/wp-json\/wp\/v2\/pages\/435\/revisions"}],"predecessor-version":[{"id":448,"href":"https:\/\/aholab.ehu.eus\/deeprestore\/en\/wp-json\/wp\/v2\/pages\/435\/revisions\/448"}],"wp:attachment":[{"href":"https:\/\/aholab.ehu.eus\/deeprestore\/en\/wp-json\/wp\/v2\/media?parent=435"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}