Synergistic drug combination screening is a promising strategy in drug discovery, but it involves navigating a costly and complex search space. While AI, particularly deep learning, has advanced synergy predictions, its effectiveness is limited by the low occurrence of synergistic drug pairs. Active learning, which integrates experimental testing into the learning process, has been proposed to address this challenge. In this work, we explore the key components of active learning to provide recommendations for its implementation. We find that molecular encoding has a limited impact on performance, while the cellular environment features significantly enhance predictions. Additionally, active learning can discover 60% of synergistic drug pairs with only exploring 10% of combinatorial space. The synergy yield ratio is observed to be even higher with smaller batch sizes, where dynamic tuning of the exploration-exploitation strategy can further enhance performance.