Currently, the energy dissipation efficiency of intermediate column dampers is extremely low, and traditional lever amplification damping systems occupy a large space in buildings. Aiming at solving these problems, this paper puts forward a new intermediate column–lever negative stiffness viscous damper (CLNVD), which has the characteristics of small impact on building space and significant amplification of the damper displacement. The CLNVD consists of the following four parts: the viscous damper, the negative stiffness device, the lever, and the intermediate column. This paper introduces the displacement amplification coefficient (fd) to assess the CLNVD’s displacement amplification effect and introduces the energy dissipation coefficient (fE) to assess the CLNVD’s energy dissipation effect. The expressions for fd and fE are derived according to the geometric magnification coefficient and effective displacement factor. Moreover, the impacts of multiple factors including the CLNVD’s position, the lever’s amplification coefficient, the bending line stiffness of beam, the negative stiffness, the damping coefficient, the damping index, and the inter-story displacement on the CLNVD’s fd and fE are elaborated. The analysis results reveal the following: when the CLNVD is located in the middle of the span, the fd and fE of the CLNVD will be maximized, and fE will increase first and then decrease as the beam’s bending line stiffness increases. Meanwhile, the amplification capability of the CLNVD increases as the lever’s amplification coefficient χ rises. When the negative stiffness does not exist, there exists an optimum lever’s amplification coefficient χ that maximizes fE. When the combination of damping coefficient c and index α satisfies a specific relationship, fE of the CLNVD reaches its largest value. When the negative stiffness and the loss stiffness of VD are within the region proposed in this paper, the CLNVD will achieve a higher fd and avoid providing significant negative stiffness. Subsequently, this paper proposes an optimization design method of the CLNVD. Finally, the amplification effect of CLNVD as well as the effectiveness of its optimization design method are verified through examples. In the case study, the CLNVD offers a larger damping ratio under the circumstance of fortification earthquakes. Under fortification and rare earthquakes, the inter-story displacement of Scheme 1 has been decreased by half roughly. According to the above-mentioned results, the CLNVD provides a brand-new approach for designers in the seismic design of buildings. Furthermore, this paper will provide beneficial reference for the damping design of other amplification devices equipped with negative stiffness dampers.