The coronavirus disease 2019 (COVID-19) pandemic has impacted the world economy in various ways. In particular, the drastic shift to telework has dramatically changed how people work. Whether the new style of working from home (WFH) will remain in our society highly depends on its effects on workers’ productivity. However, to the best of our knowledge, the effects of WFH on productivity are still unclear. By leveraging unique surveys conducted at four manufacturing firms in Japan, we assess within-company productivity differences between those who work from home and those who do not, along with identifying possible factors of productivity changes due to WFH. Our main findings are as follows. First, after ruling out the time-invariant component of individual productivity and separate trends specific to employee attributes, we find that workers who worked from home experienced productivity declines more than those who did not. Second, our analysis shows that poor WFH setups and communication difficulties are the major reasons for productivity losses. Third, we find that the mental health of workers who work from home is better than that of workers who are unable to work from home. Our result suggests that if appropriate investments in upgrading WFH setups and facilitating communication can be made, WFH may improve productivity by improving employees’ health and well-being.