Silicon photonics integrated with graphene provides a promising solution to realize integrated photodetectors operating at the communication window thanks to graphene's ultrafast response and compatibility with CMOS fabrication process. However, current hybrid graphene/silicon photodetectors suffer from low responsivity due to the weak light-graphene interaction. Plasmonic structures have been explored to enhance the responsivity, but the intrinsic metallic Ohmic absorption of the plasmonic mode limits its performance. In this work, by combining the silicon slot and the plasmonic slot waveguide, we demonstrate a novel double slot structure supporting high-performance photodetection, taking advantages of both silicon photonics and plasmonics. With the optimized structural parameters, the double slot structure significantly promotes graphene absorption while maintaining low metallic absorption within the double slot waveguide. Based on the double slot structure, the demonstrated photodetector holds a high responsivity of 603.92 mA/W and a large bandwidth of 78 GHz. The high-performance photodetector provides a competitive solution for the silicon photodetector. Moreover, the double slot structure could be beneficial to a broader range of hybrid two-dimensional material/silicon devices to achieve stronger light-matter interaction with lower metallic absorption.