To date, several microbes have been proposed as potential source-specific indicators of fecal pollution. 16S ribosomal RNA gene markers of the Bacteroidales species are the most widely applied due to their predominance in the water environment and source specificity. F-specific bacteriophage (FPH) subgroups, especially FRNA phage genogroups, are also known as potential source-specific viral indicators. Since they can be quantified by both culture-based and molecular assays, they may also be useful as indicators for estimating viral inactivation in the environment. Pepper mild mottle virus (PMMoV) and crAssphage, which are frequently present in human feces, are also potentially useful as human-specific indicators of viral pollution. This study aimed to evaluate the applicability of FPH subgroups, PMMoV, and crAssphage as indicators of source-specific fecal contamination and viral inactivation using 108 surface water samples collected at five sites affected by municipal and pig farm wastewater. The host specificity of the FPH subgroups, PMMoV, and crAssphage was evaluated by principal component analysis (PCA) along with other microbial indicators, such as 16S ribosomal RNA gene markers of the Bacteroidales species. The viabilities (infectivity indices) of FRNA phage genogroups were estimated by comparing their numbers determined by infectivity-based and molecular assays. The PCA explained 58.2% of the total information and classified microbes into three groups: those considered to be associated with pig and human fecal contamination and others. Infective and gene of genogroup IV (GIV)-FRNA phage were assumed to be specific to pig fecal contamination, while the genes of GII-FRNA phage and crAssphage were identified to be specific to human fecal contamination. However, PMMoV, infective GI-FRNA phage, and FDNA phage were suggested to not be specific to human or pig fecal contamination. FRNA phage genogroups, especially the GIV-FRNA phage, were highly inactivated in the warm months in Japan (i.e., July to November). Comparing the infectivity index of several FRNA phage genogroups or other viruses may provide further insight into viral inactivation in the natural environment and by water treatments.