Prof. Mohamad Sawan presented the latest progress of our collaborative project: The Real-Time Brain-Language Decoding

On December 30, 2024，Chair Professor Mohamad Sawan publicly presented the real-time language decoding system developed in collaboration between Westlake University and the First Affiliated Hospital of Zhejiang University School of Medicine at the "010" Pre-Incubation Technology Innovation Forum. This technology is expected to open communication pathways for aphasia patients and lead the innovation of next-generation smart medical devices.

The research team consists of interdisciplinary researchers from the CenBRAIN Neurotech Center of Excellence at Westlake University, the Natural Language Processing Laboratory at Westlake University, and the Departments of Neurosurgery and Neurology at the First Affiliated Hospital of Zhejiang University. Currently, the team has developed an intracranial EEG-Mandarin decoding system with online real-time decoding capabilities. This system is capable of single-character-level decoding and covers common Mandarin linguistic corpora. With post-processing technologies such as large language models (LLMs), the decoded character sequences are expected to achieve free sentence-level Chinese decoding, further meeting the practical application requirements of speech prostheses. The system design builds upon the team's previous research paradigm, which leverages the articulation acoustic-related patterns of Chinese syllables. By decoding intracranial EEG signals into Chinese syllable elements, the system realizes Chinese language decoding. The proposed decoding system employs a lightweight neural network-based streaming inference architecture, achieving over 200 inferences per second, with a character-level decoding delay of less than 10 milliseconds, and supporting a decoding rate of 30 Chinese characters per minute. The system also supports rapid training, deployment, and calibration. Currently, after training and calibration, multiple participants have demonstrated satisfactory decoding performance in tests involving common Chinese linguistic corpora, with the highest decoding accuracy exceeding 60%. This achievement marks a significant advancement in online neural language decoding, following the team's major breakthrough in offline neural Chinese decoding last year.

Professor Mohamad Sawan showcased our latest collaborative research project: The Real-Time Brain-Language Decoding.