Evaluation of New Online Automated Embolic Signal Detection Algorithm, Including Comparison With Panel of International Experts

Abstract

Background and Purpose-The clinical application of Doppler detection of circulating cerebral emboli will depend on a reliable automated system of embolic signal detection; such a system is not currently available. Previous studies have shown that frequency filtering increases the ratio of embolic signal to background signal intensity and that the incorporation of such an approach into an offline automated detection system markedly improved performance. In this study, we evaluated an online version of the system. In a single-center study, we compared its performance with that of a human expert on data from 2 clinical situations, carotid stenosis and the period immediately after carotid endarterectomy. Because the human expert is currently the "gold standard" for embolic signal detection, we also compared the performance of the system with an international panel of human experts in a multicenter study. Methods-In the single-center evaluation, the performance of the software was tested against that of a human expert on 20 hours of data from 21 patients with carotid stenosis and 18 hours of data from 9 patients that was recorded after carotid endarterectomy. For the multicenter evaluation, a separate 2-hour data set, recorded from 5 patients after carotid endarterectomy, was analyzed by 6 different human experts using the same equipment and by the software. Agreement was assessed by determining the probability of agreement. Results-In the 20 hours of carotid stenosis data, there were 140 embolic signals with an intensity of Ն7 dB. With the software set at a confidence threshold of 60%, a sensitivity of 85.7% and a specificity of 88.9% for detection of embolic signals were obtained. At higher confidence thresholds, a specificity Ͼ95% could be obtained, but this was at the expense of a lower sensitivity. In the 18 hours of post-carotid endarterectomy data, there were 411 embolic signals of Ն7-dB intensity. When the same confidence threshold was used, a sensitivity of 95.4% and a specificity of 97.5% were obtained. In the multicenter evaluation, a total of 127 events were recorded as embolic signals by at least 1 center. The total number of embolic signals detected by the 6 different centers was 84, 93, 108, 92, 63, and 78. The software set at a confidence threshold of 60% detected 90 events as embolic signals. The mean probability of agreement, including all human experts and the software, was 0.83, and this was higher than that for 2 human experts and lower than that for 4 human experts. The mean values for the 6 human observers were averaged to give Pϭ0.84, which was similar to that of the software. Conclusions-By using the frequency specificity of the intensity increase occurring with embolic signals, we have developed an automated detection system with a much improved sensitivity. Its performance was equal to that of some human experts and only slightly below the mean performance of a panel of human experts (Stroke. 2000;31:1335-1341.)

Automatic Classification of HITS Into Artifacts or Solid or Gaseous Emboli by a Wavelet Representation Combined With Dual-Gate TCD

“…[13][14][15][16][17][18][19][20][21][22][23][24] In our study, the signals were reviewed by 2 observers, and for inclusion, both observers had to agree that the signal was a true embolic signal. The following definition for embolic signals was used: (1) typical visible, (2) short duration, (3) high-intensity signal, (4) within the Doppler spectrum, and (5) occurring at random within the cardiac cycle.…”

“…The emboli originating from ICA stenoses are all particulate by default, because no air is able to enter the arterial system. 17,19,21,[31][32][33] Two hundred solid emboli were recorded in 24 patients (67Ϯ9 years old) admitted to the 5 neurology departments for stroke attributed to a severe ICA stenosis (Ͼ70% lumen narrowing) diagnosed by both CDFI and MR angiography, in whom no coexisting cause of stroke was found after a complete diagnostic examination, including transesophageal echocardiography. At the time of TCD embolus detection, they were taking aspirin.…”

“…12 Previous attempts at producing an automated system for classifying HITS into artifacts or emboli have failed to achieve the same level of sensitivity (embolus detection) and specificity (artifact rejection) as the human expert, considered the current "gold standard." [13][14][15][16][17][18][19][20][21][22][23][24] Without an automated embolic signal detection system that is as effective as the human expert in terms of sensitivity and specificity, embolus detection remains a time-consuming process, restricted to research studies and unsuitable for clinical practice. 25 To separate artifacts from emboli, the best compromise is dual-gate TCD, which consists of tracing the embolus at 2 different depths in the same artery (time delay) ( Figure 1).…”

“…25 To separate artifacts from emboli, the best compromise is dual-gate TCD, which consists of tracing the embolus at 2 different depths in the same artery (time delay) ( Figure 1). Lately, Cullinane et al 21 have shown that the use of a frequency-filtering approach improves the differentiation of artifacts from emboli and that the performance of an online automated system is only slightly below the mean performance of that of a panel of human experts. 21,24 Automated systems that can distinguish between solid and gaseous emboli are currently being developed.…”

“…Lately, Cullinane et al 21 have shown that the use of a frequency-filtering approach improves the differentiation of artifacts from emboli and that the performance of an online automated system is only slightly below the mean performance of that of a panel of human experts. 21,24 Automated systems that can distinguish between solid and gaseous emboli are currently being developed. 26,27 In the present study, we describe the evaluation of a new automated system for the classification of HITS into artifacts or solid or gaseous emboli, based on HITS time-frequency distribution and dual-gate TCD.…”