Sea state estimation is a fundamental problem in the development of autonomous ships. Traditional methods such as wave buoy, satellites, and wave radars are limited by locations, clouds and costs, respectively. Model-based methods are prone to incorrect estimations due to their high dependency on mathematical models of ships. As previous data-driven studies for sea state estimation only consider wave height and use the motion data from dynamic positioning vessels, this paper introduces a new, deep neural network (SSENET) to estimate sea state in light of both wave height and wave direction, and extends the generality of sensor data from ship motion with forward speed. SSENET is built on the basis of stacked convolutional neural network blocks with dense connections between different blocks, channel attention modules and a feature attention module. The dense connections build shortcut paths between input and all subsequent convolutional blocks, which can make full use of all the hierarchical features from the original time series sensor data. The channel attention modules aim to enhance the features extracted by each convolution block. The feature attention module focuses on combining the feature fusion of hierarchical features in an adaptive manner. Benchmark experiments show the competitive performance against state-ofthe-art approaches. Applying the SSENET on two datasets of zigzag motion for comparative studies shows the effectiveness of the proposed method.