A conventional hybrid circuit breaker (HCB) is used to protect a voltage source converter-based high voltage direct current transmission system (VSC-HVDC) from a short circuit fault. With the increased converter capacity, the DC protection equipment also requires a regular upgrade. This paper adopts a novel type of HCB with a fault current limiter circuit (FCLC), and focuses on the responses of voltage and current during DC faults, which are associated with parameter selection. PSCAD/EMTDC based simulation of a three-terminal VSC-HVDC system confirms the effectiveness and value of HCB with FCLC, by using an equivalent circuit modelling approach. Laboratory experimental tests validate the simulation results. The peak fault current is reduced according to the current limiting inductor (CLI) increase, and can be isolated more quickly. By adopting parallel metal oxide arrester (MOA) with the main branch of HCB, voltage stresses across the breaker components decrease during transient and continuous operation, and less energy needs to be dissipated by the MOA. The remnant current for all cases is transmitted to power dissipating resistor (PDR) in the final stage, and the fault current is reduced to the lowest possible value. When the current from the main branch is transferred to the FCLC branch, transient voltage spikes occur, while smaller PDR is required to absorb current in the final stage.