Among Ugandan fisherfolk, HIV prevalence (with estimates ranging from 15 to 40 %) is higher than in the general population (about 7 %), potentially due to high-risk behaviors and low access to HIV testing and health-care. We conducted semi-structured interviews on barriers to linkage to care with 10 key stakeholders and 25 fisherfolk within 1–2 months of their testing HIV-positive at clinic outreach events in Ugandan Lake Victoria communities. Interviews were recorded, transcribed, translated, and coded using grounded theory methods. Participants cited low healthcare access and quality of care, mobility, competing needs for work during clinic hours, stigma, and low social support as barriers. Over 10 % of clients screened positive for HIV at outreach events, and only half accessed care. Linkage to care issues may begin with the failure to attract high-risk fisherfolk to testing. New models of HIV testing and treatment delivery are needed to reach fisherfolk.