The COVID-19 pandemic has brought forth a novel and challenging investment environment for stock markets in both developed and emerging economies. This study delves into the under-researched field of factor investing, analysing the influence of firm-specific factors on stock performance in sub-Saharan Africa during the pandemic in four selected stock exchanges. Using explainable artificial intelligence (XAI), we analysed the cross-sectional data of firms listed on these exchanges in both the pre-pandemic and pandemic periods to identify the key factors driving stock returns. Our findings reveal that quality factors such as high profitability and low leverage emerged as the dominant drivers of resilience during the pandemic. This contrasts with the pre-pandemic scenario, where traditional factors, such as size, momentum, and value, were the primary determinants. We also find that during the pandemic, small-cap stocks generally outperformed large-cap stocks on larger stock exchanges, whereas large-cap stocks outperformed on smaller stock exchanges. The pandemic also ushered in a shift in the norm, as we found that highly volatile stocks outperformed low-volatility stocks and stocks classified as winners in the pre-pandemic period experienced a momentum crash. Further analysis explores the potential of factor-based portfolios. We demonstrate that portfolios constructed based on the identified key factors consistently outperform their market-cap weighted counterparts across all exchanges, both pre- and post-pandemic. This research highlights the importance of focusing on firm-specific factors to construct more resilient investment portfolios and potentially attain superior returns, even during times of economic turbulence.