Depression is a widely prevalent psychiatric illness with variable levels of severity that necessitate different approaches to treatment. To enhance the management of this condition, there is a growing interest in utilizing mobile devices, especially smartphones, for remote monitoring of patients. This study aims to build prediction models for depression severity based on active and passive features collected from patients with major depressive disorder (MDD) and healthy controls to assess the feasibility of remote monitoring of depression severity. Using data from 142 participants (85 healthy controls, 67 MDD) we extracted features such as GPS-derived mobility markers, ecological momentary assessments (EMA), age, and sex to develop machine learning models of depression severity on the different diagnostic subgroups in this cohort. Our results indicate that the employed models outperformed baseline estimators in random split scenarios. However, the improvement was marginal in user-split scenarios, highlighting the need for larger and more diverse samples for clinical utility. Among the features, mood EMA emerged as the most influential predictor, followed by GPS-derived mobility features. Models also showed a significant association between depression severity and average reported mood, as well as GPS-derived mobility markers such as number of places visited and percent home. While predicting composite depression scores is important, future studies could explore predicting individual symptom items or symptom groups for a more comprehensive assessment of depression severity. Challenges for clinical utility include participant dropout, which could be addressed through more engaging app design to promote user adherence. Harmonization of phone-derived measures is also crucial to facilitate model transfer across studies. In conclusion, this study contributes valuable evidence supporting the potential utility of smartphone data for mood state monitoring and predicting depression severity. Future research should focus on predicting depression further ahead in time and addressing the challenges identified to create more robust and effective depression monitoring solutions using smartphone-based data.