Background After decades of increased opioid pain reliever prescribing, providers are rapidly reducing prescribing. We hypothesized that reduced access to prescribed opioid pain relievers among patients previously reliant upon opioid pain relievers would result in increased illicit opioid use. Methods and findings We conducted a retrospective cohort study among 602 publicly insured primary care patients who had been prescribed opioids for chronic non-cancer pain for at least three consecutive months in San Francisco, recruited through convenience sampling. We conducted a historical reconstruction interview and medical chart abstraction focused on illicit substance use and opioid pain reliever prescriptions, respectively, from 2012 through the interview date in 2017-2018. We used a nested-cohort design, in which patients were classified, based on opioid pain reliever dose change, into a series of nested cohorts starting with each follow-up quarter. Using continuation-ratio models, we estimated associations between opioid prescription discontinuation or 30% increase or decrease in dose, relative to no change, and subsequent frequency of heroin and non-prescribed opioid pain reliever use, separately. Models controlled for demographics, clinical and behavioral characteristics, and past use of heroin or non-prescribed opioid pain relievers. A total of 56,372 and 56,484 participant-quarter observations were included from the 597 and 598 participants available for analyses of heroin and non-prescribed opioid pain reliever outcomes, respectively. Participants discontinued from prescribed opioids were more likely to use heroin (Adjusted Odds Ratio (AOR) = 1.57, 95% CI: 1.25-1.97) and non-prescribed opioid pain relievers (AOR = 1.75, 1.45-2.11) more frequently in subsequent quarters compared to participants with unchanged opioid prescriptions. Participants whose opioid pain reliever dose increased were more likely to use heroin more frequently (AOR = 1.67, 1.32-2.12). Results held throughout sensitivity analyses. The main limitations were the observational nature of results and limited generalizability beyond safety-net settings.