Validation of artificial intelligence for epileptic seizure detection: Moving from research into the clinic
Year: 2024
Type: editorial
Abstract: Epilepsy diagnosis, monitoring, and treatment decisions currently depend heavily on several-minute-long electroencephalography (EEG) recordings, whose effectiveness is limited by their short and episodic nature.1 This limitation could be overcome with the advent of wearable long-term EEG technology and appropriate automated analysis of recorded data for more accurate and timely detection of seizures. Automated epileptic seizure detection algorithms have been developed since the 1980s, and hundreds of scientific papers have been published since then. These algorithms have multiple intended uses, including assisting clinical teams in identifying subtle seizures (such as focal impaired awareness seizures) to ensure timely intervention and care. But despite strong interest and advancements in research, only a few algorithms have made it to the clinic, and currently they are not widely used. A recent survey highlighted that most clinicians would be interested in using EEG-based automated seizure detection algorithms.2 About 70% of surveyed medical professionals responded that they would use seizure detection tools for all seizure types, including non-motor seizures, which they experienced as the most difficult to manage without continuous EEG readings. However, they pointed out that current algorithms suffer from poor performance, lack of generalizability, and unproven replicability. These issues are common in artificial intelligence domains where validation standards are not yet well established, as is the case with automated epileptic seizure detection. Evaluating and comparing these algorithms is extremely difficult due to a lack of standardization in evaluation data sets, definition of the task, and of performance metrics. We therefore need a framework for the validation of seizure detection algorithms that builds on existing standards and guidelines. Such a framework should ideally propose a standardized input data format for seizure detection algorithms. This format can, for example, comply with recent efforts to standardize brain imaging studies such as the Brain Imaging Data Structure (BIDS),3 along with recent work to standardize clinical annotations of EEG files: SCORE.4 The format should allow algorithms to operate on any compliant EEG data set. This is currently being discussed within the epilepsy community, and an international interdisciplinary group is working on an open-source framework for the validation of EEG-based automated seizure detection algorithms (SzCORE, https://arxiv.org/pdf/2402.13005). This framework defines data formats, tasks, and performance metrics that are tailored for both developers of algorithms and clinicians. A remaining challenge is to encourage the adoption of a framework such as SzCORE. Wong et al. suggested a five-component guideline for seizure detection system for adoption, emphasizing performance, explainability, generalizability, replicability, and clinical utility.2 Access is likely also crucial, through open-source tools that implement the different components of the framework and through a platform to allow algorithm developers to have their own algorithm independently evaluated on available data sets, which include paediatric epilepsy. This platform would additionally enable benchmarking of algorithms on the same data, using a common definition of the seizure detection task and reporting performance on common performance metrics. This would effectively solve the issues that are currently experienced as challenges in the evaluation of algorithms, and it should provide the necessary tools to build trust, and generalizability and reproducibility of results, thereby facilitating the adoption of the technology in clinical centres. Not required.
DOI: 10.1111/dmcn.16002
Author: Jonathan Dan
Institution: École Polytechnique Fédérale de Lausanne
Language: English
Subfield: Cognitive Neuroscience
Field: Neuroscience
Domain: Life Sciences
Open Access status: bronze
OpenAlex ID: W4399867788