The agglomeration problem of nanofillers, for instance, carbon nanotubes (CNTs) in a poly(ether ether ketone) (PEEK) matrix, is still a challenging assignment due to the intrinsic inert nature of PEEK to organic solvents. In this work, organically modified montmorillonite (MMT) was introduced as a second filler for improving the dispersion of CNTs in the PEEK matrix and enhancing the mechanical properties, as well as reducing the cost of the materials. The nanocomposites were fabricated through melt-mixing PEEK with CNTs/MMT hybrids, which were prepared in advance by mixing CNTs with MMT in water. The introduction of MMT improved the dispersion stability of CNTs, as characterized by sedimentation and zeta potential. The CNTs/MMT hybrids were maintained in PEEK nanocomposites as demonstrated by the transmission electron microscope. The mechanical and thermomechanical measurements revealed that CNTs together with MMT had a strong reinforcement effect on the PEEK matrix, especially at high temperature, although it had a negative effect on the toughness. A maximum increase of 48.1% was achieved in storage modulus of PEEK nanocomposites with 0.5 wt% CNTs and 2 wt% MMT at 240 °C, compared to that of neat PEEK. The differential scanning calorimetry results revealed that CNTs accelerated the crystallization of the PEEK matrix while a further addition of MMT played an opposite role. The nucleation activity of the fillers was also evaluated by the Dobreva method.