This paper presents a deep framework EddyDet to automatically detect oceanic eddies in Synthetic Aperture Radar (SAR) images. The EddyDet has been developed using the Mask Region with Convolutional Neural Networks (Mask RCNN) framework, incorporating two new branches: Edge Head and Mask Intersection over Union (IoU) Head. The Edge Head can learn internal texture information implicitly, and the Mask IoU Head improves the quality of predicted masks. A SAR dataset for Oceanic Eddy Detection (SOED) is specifically constructed to evaluate the effectiveness of the EddyDet model in detecting oceanic eddies. We demonstrate that the EddyDet is capable of achieving acceptable eddy detection results under the condition of limited training samples, which outperforms a Mask RCNN baseline in terms of average precision. The combined Edge Head and Mask IoU Head have the ability to describe the characteristics of eddies more correctly, while the EddyDet shows great potential in practice use accurately and time efficiently, saving manual labor to a large extent.