Evasive Spectre attacks have used additional nop or memory delay instructions to make effective hardware performance counter based detectors with lower attack detection successful rate. Interestingly, the detection performance gets worse under different workloads. For example, the attack detection successful rate is only 59.8% for realistic applications, while it is much lower 27.52% for memory stress test. Therefore, this paper proposes a two-stage smart detector T-Smade designed for evasive Spectre attacks (e.g., evasive Spectre nop and evasive Spectre memory) under various workloads. T-Smade uses the first-stage detector to identify the type of workloads and then selects the appropriate second-stage detector, which uses four hardware performance counter events to characterize the high cache miss rate and low branch miss rate of Spectre attacks. More importantly, the second stage detector adds one dimension of reusing cache miss rate and branch miss rate to exploit the characteristics of various workloads to detect evasive Spectre attacks effectively. Furthermore, to achieve the good generalization for more unseen evasive Spectre attacks, the proposed classification detector T-Smade is trained by the raw data of Spectre attacks and non-attacks in different workloads using simple Multi-Layer Perception models. The comprehensive results demonstrate that T-Smade makes the average attack detection successful rate of evasive Spectre nop under different workload return from 27.52% to 95.42%, and that of evasive Spectre memory from 59.8% up to 100%.