Nowadays, the Overcurrent (OC) and Earth Fault (EF) relays coordination problem is one of the most complex and challenging concerns of power protection and network operators due to the high and volatile generation capacity of renewable energy sources in the grid. In this article, a new and dynamic optimal coordination scheme based on a novel hybrid tripping characteristic has been designed and developed for Over Current Relays (OCRs). Considering the impact of renewable energy sources such as the photovoltaic (PV) system on fault characteristic, this work presents and verifies a novel dynamic and hybrid tripping to achieve minimum tripping time and improve the OCR and EF relays coordination performance in terms of security, sensitivity, and selectivity. The proposed dynamic and hybrid scheme will help the OCRs to cover the EF events, and it has been tested under different fault scenarios compared to the literature. The IEEE-9 and IEEE-33 bus systems are implemented in the ETAP package to validate the effectiveness of the proposed hybrid characteristics against traditionally well-established IEC characteristics. Furthermore, the performance of the proposed advance and dynamic protection approach which doesn't require a communication infrastructure is investigated for a power network with PV plants under different grid operation modes and topology to provide more robustness protection system. The results, as presented using Industrial software (ETAP), showed that the novel dynamic and hybrid tripping scheme improved the speed of the total time tripping different fault scenarios and location by more than 50% and covers all EF events compared to traditional OCR schemes from the literature. The proposed novel dynamic approach has superior performance in detecting high-impedance faults and significantly reducing the tripping time on the IEEE 33 bus network by 47%.INDEX TERMS Over current relays, Earth fault relay, Dynamic coordination scheme, Renewable energy sources, PV.