In digital manipulation, creating fake images/videos or swapping face images/videos with another person is done by using a deep learning algorithm is termed deep fake. Fake pornography is a harmful one because of the inclusion of fake content in the hoaxes, fake news, and fraud things in the financial. The Deep Learning technique is an effective tool in the detection of deep fake images or videos. With the advancement of Generative adversarial networks (GAN) in the deep learning techniques, deep fake has become an essential one in the social media platform. This may threaten the public, therefore detection of deep fake images/videos is needed. For detecting the forged images/videos, many research works have been done and those methods are inefficient in the detection of new threats or newly created forgery images or videos, and also consumption time is high. Therefore, this paper focused on the detection of different types of fake images or videos using Fuzzy Fisher face with Capsule dual graph (FFF-CDG). The data set used in this work is FFHQ, 100K-Faces DFFD, VGG-Face2, and Wild Deep fake. The accuracy for FFHQ datasets, the existing and proposed systems obtained the accuracy of 81.5%, 89.32%, 91.35%, and 95.82% respectively.