As the newly built tokamak in China, HL-3 is going to explore high performance operation scenario such as super H-mode. The energy confinement and core parameters in super H-mode can be much larger than that in normal H-mode. Based on the pedestal simulation code EPED, the operation space of super H-mode is obtained in HL-3. Magnetic shear decreases with increasing triangularity, and then the super H-mode can be achieved. The threshold of triangularity for accessing super H-mode in HL-3 is around 0.4. By using BOUT++, nonlinear simulation study of the pedestal instabilities in super H-mode equilibrium is executed for the first time. As expected, low n peeling mode which can cause much energy loss (17%) from the pedestal region is dominant in super H-mode. Such a big collapse at pedestal region will lead the transition of super H-mode to H-mode. It is crucial to expand the parameter space of the super H-mode or mitigate the ELM size for sustaining the super H-mode operation. E×B velocity shear is found to play an important role for controlling the ELMs in HL-3. On the one hand, small E×B velocity shear leads to large growth rate but small ELM size around peeling boundary. ELM size is closely related with both of growth rate of peeling-ballooning mode and duration time of linear phase. On the other hand, large E×B velocity shear can stabilize the instabilities near ballooning boundary, and then the parameter space of super H-mode will be enlarged.