Viruses can infect the brain in individuals with and without HIV-infection: however, the brain virome is poorly characterized. Metabolic alterations have been identified which predispose people to substance use disorder (SUD), but whether these could be triggered by viral infection of the brain is unknown. We used a target-enrichment, deep sequencing platform and bioinformatic pipeline named “ViroFind”, for the unbiased characterization of DNA and RNA viruses in brain samples obtained from the National Neuro-AIDS Tissue Consortium. We analyzed fresh frozen post-mortem prefrontal cortex from 72 individuals without known viral infection of the brain, including 16 HIV+/SUD+, 20 HIV+/SUD-, 16 HIV-/SUD+, and 20 HIV-/SUD-. The average age was 52.3 y and 62.5% were males. We identified sequences from 26 viruses belonging to 11 viral taxa. These included viruses with and without known pathogenic potential or tropism to the nervous system, with sequence coverage ranging from 0.03 to 99.73% of the viral genomes. In SUD+ people, HIV-infection was associated with a higher total number of viruses, and HIV+/SUD+ compared to HIV-/SUD+ individuals had an increased frequency of Adenovirus (68.8 vs 0%; p<0.001) and Epstein-Barr virus (EBV) (43.8 vs 6.3%; p=0.037) as well as an increase in Torque Teno virus (TTV) burden. Conversely, in HIV+ people, SUD was associated with an increase in frequency of Hepatitis C virus, (25 in HIV+/SUD+ vs 0% in HIV+/SUD-; p=0.031). Finally, HIV+/SUD- compared to HIV-/SUD- individuals had an increased frequency of EBV (50 vs 0%; p<0.001) and an increase in TTV viral burden, but a decreased Adenovirus viral burden. These data demonstrate an unexpectedly high variety in the human brain virome, identifying targets for future research into the impact of these taxa on the central nervous system. ViroFind could become a valuable tool for monitoring viral dynamics in various compartments, monitoring outbreaks, and informing vaccine development.