연속 음성 유발 전위 기반 언어인지도 예측 모델 개발

Abstract

This study aimed to develop the deep learning model to assess speech intelligibility (SI) objectively without attention from continuous speech-evoked potential (CSEP). The CSEP extracted in this study was the temporal response function of neural tracking. The neural tracking here is a mathematical approach to quantify how well the entrained activity is aligned to speech features. While a popular speech feature for neural tracking is the speech envelope, phoneme information is crucial to understand the speech. In Chapter II, the phoneme onset time as an event cue was used for phoneme-based neural tracking. The phoneme CSEP was validated using the natural and 4-channel vocoded conditions. The CSEP using phoneme onset neural tracking revealed SI differences at the N1-P2 complex. In other words, phoneme onset time can represent the degree of speech intelligibility. SI prediction model was developed with speech features of temporal envelope and phoneme information. The SI prediction deep learning model was trained using the features of ERPs, envelope-based CSEPs (ENV), phoneme-based CSEPs (PH), or phoneme-envelope-based CSEPs (PHENV) with the output of behavioral speech intelligibility scores. Data augmentation algorithm was employed to encourage the number of the training dataset. The validation loss of all models decreased during the first two training epochs and saturated thereafter. The deep learning models were no over-fitted problem. The performances of models were 97.34 (ERP), 99.05 (ENV), 99.87 (PH), and 99.97 % (PHENV), which are comparable to the random chance level of 2.63 %. The results demonstrated that the SI prediction with CSEP could precisely assess speech intelligibility. In addition, the informative electrodes were estimated by using Occlusion sensitivity map. The informative electrodes were language dominant area in the PH and PHENV model.