The robot control with safety consideration is required since robots and human work together in the same space more frequently in these days. For safety, robots must have compliance to human force and robust tracking performance with high impednace for the nonhuman disturbances. The novel idea is proposed to achieve the compliance and high impedance with one controller structure. For the compliance, the ISMC(Integral Sliding Mode Control) and HDOB(Human Disturbance Observer) The human force is identified by using the human band pass filter and its output is sent to the sliding surface. The sliding mode dynamic is affected by human disturbance and the compliance for human is achieved. The disturbances besides human frequencies are decoupled by the ISMC and the robust tracking is achieved. The additional LDOB(Low Frequency Disturbance Observer) decreases the maxim nonlinear gain and leads low chattering. The introduction of human disturbance into the sliding mode dynamic is the main novel idea of this paper.