Background: Agriculture remains to be the leading sector that contributes enormously to economic development in Ethiopia. Despite its significant contribution to livelihoods, the sector faces persistent challenges due to depletion of natural resources and soil erosion that resulted in diminishing crop and livestock productivity. In order to curb the effects of land degradation, the Government of Ethiopia has been taking serious measures to expand Soil and Water Conservation (SWC) practices throughout the country. Despite the efforts made, the adoption of new practices by farmers have been generally low. This study was aimed at assessing factors influencing smallholder farmers’ decision on the use of improved structural SWC practices in Haramaya district, eastern Ethiopia. A multi-stage sampling technique was used to select 120 farm households and 248 plots. A structured interview schedule was used to collect primary data. Descriptive and inferential statistics and Multinomial Logit (MNL) regression model were used to analyze the data. Results: The key findings showed that a host of socio-demographic, economic and institutional factors significantly affected smallholders’ decision to adopt improved structural SWC. In this study, we found that education, farming experience, plot area, distance of the plot from dwelling unit, number of economically active household members, and extension contact were the significant predictors of using improved SWC structures. Conclusions: Based on our findings, we concluded that improved SWC measures should be scaled up through a concerted effort of extension workers, local administration and other relevant non-state actors. In particular, the extension system should encourage rural communities on sustainable management and use of natural resources. Moreover, the need to create learning opportunities through facilitating appropriate educational and training programs for farmers and focusing on proper management of available economically active household members should be emphasized.