The infant mortality rate remains unacceptably high in sub-Saharan African countries. Ethiopia has one of the highest rates of infant death. This study aimed to identify individual-and community-level factors associated with infant death in the rural part of Ethiopia. The data for the study was obtained from the 2016 Ethiopian Demographic and Health Survey. A total of 8667 newborn children were included in the analysis. The multilevel logistic regression model was considered to identify the individual and community-level factors associated with new born mortality. The random effect model found that 87.68% of the variation in infant mortality was accounted for by individual and community level variables. Multiple births (AOR = 4.35; 95%CI: 2.18, 8.69), small birth size (AOR = 1.29; 95%CI: 1.10, 1.52), unvaccinated infants (AOR = 2.03; 95%CI: 1.75, 2.37), unprotected source of water (AOR = 1.40; 95%CI: 1.09, 1.80), and non-latrine facilities (AOR = 1.62; 95%CI: 1.20) were associated with a higher risk of infant mortality. While delivery in a health facility (AOR = 0.25; 95%CI: 0.19, 0.32), maternal age 35–49 years (AOR = 0.65; 95%CI: 0.49, 0.86), mothers receiving four or more TT injections during pregnancy (AOR = 0.043, 95% CI: 0.026, 0.071), and current breast feeders (AOR = 0.33; 95% CI: 0.26, 0.42) were associated with a lower risk of infant mortality. Furthermore, Infant mortality rates were also higher in Afar, Amhara, Oromia, Somalia, and Harari than in Tigray. Infant mortality in rural Ethiopia is higher than the national average. The government and other concerned bodies should mainly focus on multiple births, unimproved breastfeeding culture, and the spacing between the orders of birth to reduce infant mortality. Furthermore, community-based outreach activities and public health interventions focused on improving the latrine facility and source of drinking water as well as the importance of health facility delivery and received TT injections during the pregnancy.