BackgroundCancer and its treatment can result in psychological distress in both adults with cancer and in their family caregivers. This psychological distress acts as a significant adverse factor in patient-caregiver dyads. The study purposes included: (i) to assess anxiety and depression in adults with cancer and their family caregivers, and examine the dyadic relationship of anxiety and depression in patient-caregiver dyads; (ii) to investigate factors that may modify these relationships; and (iii) to explore the impact of anxiety and depression on patient-caregiver dyad quality of life (QOL).MethodsThis was a secondary analysis of a cross-sectional study. Participants consisted of 641 patient-caregiver dyads. Participants completed a survey assessing adults with cancer-related, family caregiver-related, and family-related variables using a demographic/clinical information sheet. In addition, anxiety/depression and QOL were assessed by using the Chinese version of the Hospital Anxiety and Depression Scale and SF-12 respectively. Data were analyzed by using descriptive statistics, Pearson correlations, subgroup analysis, and the Actor-Partner Interdependence Model.ResultsNearly one-third of participants had experienced anxiety and depression. Adults with cancer and family caregivers experienced a similar degree of anxiety and depression. Correlations (r) of anxiety and depression between patient-caregiver dyads ranged from 0.25 to 0.32. Various factors influencing the anxiety and depression relationship between patient-caregiver dyads were identified, including adults with cancer-related (e.g., age, gender, marital status, level of being informed about the disease, different types of cancer and treatment), family caregiver-related (e.g., being the spouse of a patient, duration in their role as a family caregiver, and amount of time spent on caregiving each day), and family-related (family relationship pre- and post-cancer, financial burden on the family due to cancer treatment) variables. To some extent, both actor and partner effects were identified for anxiety and depression on the QOL of patient-caregiver dyads.ConclusionsStudy findings call attention to anxiety and depression, as well as related factors, in patient-caregiver dyads. The underlined essential components and focus of intervention, which will be developed to decrease psychological distress and improve QOL in patient-caregiver dyads, included individual characteristics of patient-caregiver dyads, family relationship, and anxiety and depression in their counterparts.Electronic supplementary materialThe online version of this article (10.1186/s12955-018-1051-3) contains supplementary material, which is available to authorized users.