Magnetic sensors actuated by bulk acoustic wave (BAW) have attracted extensive attention due to the fact of their high sensitivity, GHz-level high frequency, and small size. Different from previous studies, suppression of energy loss and improvement in energy conversion efficiency of the BAW magnetoelectric (ME) sensor were systematically considered during the device design in this work. Finite element analysis models of material (magnetic composite), structure (ME heterostructure), and device (BAW ME magnetic sensor) were established and analyzed in COMSOL software. Additionally, the magnetic composite was prepared by radio frequency magnetron sputtering, and its soft magnetism was characterized by magnetic hysteresis loop and surface roughness. The research results demonstrate that after inserting four layers of 5 nm Al2O3 films, a performance of 86.7% eddy current loss suppression rate, a less than 1.1% magnetostriction degradation rate, and better soft magnetism were achieved in 600 nm FeGaB. Furthermore, compared with other structures, the two-layer piezomagnetic/piezoelectric heterostructure had a better ME coupling performance. Eventually, the design of the BAW ME magnetic sensor was optimized by the resonance-enhanced ME coupling to match the resonance frequency between the magnetic composite and the BAW resonator. When a 54,500 A/m direct current bias magnetic field was applied, the sensor worked at the first-order resonance frequency and showed good performance. Its linearity was better than 1.30%, the sensitivity was as high as 2.33 μmV/A, and the measurement range covered 0–5000 A/m.