Several genome-wide CRISPR knockout screens have been conducted to identify host factors regulating SARS-CoV-2 replication, but the models used have often relied on overexpression of ACE2 receptor. Additionally, such screens have yet to identify the protease TMPRSS2, known to be important for viral entry at the plasma membrane. Here, we conducted a meta-analysis of these screens and showed a high level of cell-type specificity of the identified hits, arguing for the necessity of additional models to uncover the full landscape of SARS-CoV-2 host factors. We performed genome-wide knockout and activation CRISPR screens in Calu-3 lung epithelial cells, as well as knockout screens in Caco-2 intestinal cells. In addition to identifying ACE2 and TMPRSS2 as top hits, our study reveals a series of so far unidentified and critical host-dependency factors, including the Adaptins AP1G1 and AP1B1 and the flippase ATP8B1. Moreover, new anti-SARS-CoV-2 proteins with potent activity, including several membrane-associated Mucins, IL6R, and CD44 were identified. We further observed that these genes mostly acted at the critical step of viral entry, with the notable exception of ATP8B1, the knockout of which prevented late stages of viral replication. Exploring the pro- and anti-viral breadth of these genes using highly pathogenic MERS-CoV, seasonal HCoV-NL63 and -229E and influenza A orthomyxovirus, we reveal that some genes such as AP1G1 and ATP8B1 are general coronavirus cofactors. In contrast, Mucins recapitulated their known role as a general antiviral defense mechanism. These results demonstrate the value of considering multiple cell models and perturbational modalities for understanding SARS-CoV-2 replication and provide a list of potential new targets for therapeutic interventions.