The rapid development of the Internet and the wide spread of its applications has affected many aspects of our life. However, this development also makes the cyberspace more vulnerable to various attacks. Thus, detecting and preventing these attacks are crucial for the next development of the Internet and its services. Recently, machine learning methods have been widely adopted in detecting network attacks. Among many machine learning methods, AutoEncoders (AEs) are known as the state-of-the-art techniques for network anomaly detection. Although, AEs have been successfully applied to detect many types of attacks, it is often unable to detect some difficult attacks that attempt to mimic the normal network traffic. In order to handle this issue, we propose a new model based on AutoEncoder called Double-Shrink AutoEncoder (DSAE). DSAE put more shrinkage on the normal data in the middle hidden layer. This helps to pull out some anomalies that are very similar to normal data. DSAE are evaluated on six well-known network attacks datasets. The experimental results show that our model performs competitively to the state-of-the-art model, and often out-performs this model on the attacks group that is difficult for the previous methods.