The nuclear accident that accompanied the Great East Japan Earthquake of 11 March, 2011, was also an information disaster. A serious problem that arose after the accident and persisted for a long time was the damage caused by harmful rumors (DCBHR). In 2016, a cross-sectional questionnaire survey on health and information was conducted in Fukushima. The eligible population of this survey was 2000 Fukushima residents, which included those in the evacuated areas. We received 861 responses. Data were analyzed using the responses to the question about perceived DCBHR as the objective variable and the sources of information residents trusted and the media they used as explanatory variables. A multiple logistic regression analysis revealed that those who trusted government ministries and local commercial TV were significantly associated with no effect. In contrast, those who used Internet sites and blogs were significantly associated with a negative effect. This study underlines the pivotal importance of media and information, literacy, and education and discusses how these should be improved to avoid DCBHR in the future. Furthermore, accurate information should be made available to all sections of the population to diminish DCBHR.