Course Project • Highlights

Speech Onset Detection Using sEEG | CPSC 554X Course Project

  • python
  • keras
  • scikit-learn
  • librosa
  • 2022

The goal of this course project was to develop a deep learning model for detecting speech onset, providing potential applications for brain-computer interfaces and assisting individuals with speaking disabilities.

We used a public dataset of stereotactic electroencephalography (sEEG) signals and the corresponding audio from 10 patients speaking prompted Dutch words. (Link: https://www.nature.com/articles/s41597-022-01542-9)

For feature extraction, we processed the audio and manually labeled the timestamps for each speech onset (when the patient would begin speaking).

image.png

We also extracted frequency-based features from the sEEG signals and clustered channels using spacial similarity via Bisecting K-Means.

We developed a CNN-LSTM-based model which achieved an AUC of 0.89, outperforming classical ML models such and Random Forest.

image.png