Humans can be infected by zoonotic avian, pandemic and seasonal influenza A viruses (IAVs), which differ by receptor specificity and conformational stability of their envelope glycoprotein hemagglutinin (HA). It was shown that receptor specificity of the HA determines the tropism of IAVs to human airway epithelial cells, the primary target of IAVs in humans. Less is known about potential effects of the HA properties on viral attachment, infection and activation of human immune cells. To address this question, we studied the infection of total human peripheral blood mononuclear cells (PBMCs) and subpopulations of human PBMCs with well characterized recombinant IAVs differing by the HA and the neuraminidase (NA) but sharing all other viral proteins. Monocytes and all subpopulations of lymphocytes were significantly less susceptible to infection by IAVs with avian-like receptor specificity as compared to human-like IAVs, whereas plasmacytoid dendritic cells (pDCs) and myeloid dendritic cells were equally susceptible to IAVs with avian-like and human-like receptor specificity. This tropism correlated with the surface expression of 2-3-linked sialic acids (avian-type receptors) and 2-6-linked sialic acids (human-type receptors). Despite a reduced infectivity of avian-like IAVs for PBMCs, these viruses were not less efficient than human-like IAVs in terms of cell activation as judged by the induction of cellular mRNA of IFN-α, CCL5, RIG-I, and IL-6. Elevated levels of IFN-α mRNA were accompanied by elevated IFN-α protein secretion in primary human pDC. We found that high basal expression in monocytes of antiviral interferon-induced transmembrane protein 3 (IFITM3) limited viral infection in these cells. siRNA-mediated knockdown of IFITM3 in monocytes demonstrated that viral sensitivity to inhibition by IFITM3 correlated with the conformational stability of the HA. Our study provides new insights into the role of host- and strain-specific differences of HA in the interaction of IAVs with human immune cells and advances current understanding of the mechanisms of viral cell tropism, pathogenesis and markers of virulence.