Background People report using cannabis as a substitute for prescription medications but may be doing so without the knowledge of their primary health care providers (PCPs). This lack of integration creates serious concerns, e.g., using cannabis to treat medical conditions that have established treatment options. Methods We conducted an anonymous, cross-sectional online survey among patrons of a medical cannabis dispensary in Michigan (n = 275) to examine aspects of their relationship with their PCP and their perceptions of PCP knowledge related to cannabis. Results Overall, 64% of participants initiated medical cannabis use based on their own experiences vs. 24% citing advice from their PCP. Although 80% reported that their PCP knew they currently used medical cannabis, 41% reported that their PCP had not always known. Only 14% obtained their medical cannabis authorization from their PCP. Only 18% of participants rated their PCP’s knowledge about medical cannabis as very good or excellent and only 21% were very or completely confident in their PCP’s ability to integrate medical cannabis into their treatment. Although 86% had substituted cannabis for pharmaceutical medications, 69% (n = 134) of those who substituted reported some gap in their PCP’s knowledge of their substitution, and 44% (n = 86) reported that their PCP was currently unaware of their substitution. Conclusions Patients frequently substitute cannabis for prescription drugs, often without PCP knowledge. Although most participants disclosed cannabis use to their PCP, their perceptions of PCP knowledge ranged widely and many obtained medical cannabis licensure from an outside physician. Our results highlight the need for standardized physician education around appropriate medical cannabis use.