Given the pervasive role of smartphones in modern life, research into their impact on well-being has flourished. This study addresses methodological shortcomings of previous research by using smartphone-log data and Ecological Momentary Assessments (EMA) to explore the bidirectional within-person relationship between smartphone usage and momentary well-being variables (i.e., affect valence, loneliness, positive affect, and negative affect). We further examine different categories of smartphone usage, namely Communication, Social Media, and Other app usage. We analyze three samples (N1 = 225, N2 = 17, N3 = 13; with T1 = 7874, T2 = 2566, and T3 = 1533 EMA reports) with multilevel models to test our preregistered hypotheses.Our results suggest that smartphone usage within an hour before EMA assessment, especially using Social Media apps, is associated with reduced affect valence and increased loneliness on a within-person level. Loneliness is associated with heightened smartphone usage than usual, particularly the use of Social Media apps, within the hour following EMA assessments. While our results suggest that the associations between smartphone usage and affect may be weak (range standardized β = .00–.04), the bidirectional association with loneliness is more pronounced (range standardized β = .04– .09). While temporary reductions in smartphone usage, particularly social media use, may alleviate feelings of loneliness, further research employing intervention designs is warranted to establish experimental evidence.