Purpose Identification of potential causes of academic difficulties and unsatisfactory student performance is essential for any educational institution. This study has been undertaken to identify risk factors that are associated with academic difficulties among medical students by exploring lifestyle and social factors, health problems, study and exam habits and psychological status of students at the faculty of medicine in Jazan University, Saudi Arabia. Materials and Methods This observational analytical case–control study was performed at the Faculty of Medicine at Jazan University to highlight the factors that influence the academic performance of medical students. We studied 40 students with academic difficulties (cases) and 40 controls based on their grade point average. Cases were matched to controls according to sex, age, and medical year. Results Several factors seem to negatively affect the students’ academic performance; students with academic difficulties are older, require more time to reach the college, their perceived English proficiency is less than their counterparts, and they spend less time socializing. On the other hand, having a strong family support system, socializing with friends, access to the internet, and engagement in extracurricular activities seems to positively affect the students’ performance. No difference was observed in study habits or sleep quality. The prevalence of anxiety, stress, and depression were all low for both cases and controls. Conclusion We have identified many factors, most of which are related to the student’s lifestyle and social habits, that positively and negatively affect academic performance.