Genome-wide association studies have reported >100 risk loci for rheumatoid arthritis (RA). These loci have been shown to be enriched in immune cell-specific enhancers, but analysis so far has excluded stromal cells, such as synovial fibroblasts (FLS), despite their crucial involvement in the pathogenesis of RA. Here we integrated DNA architecture (ChIP-seq), 3D chromatin interactions (HiC, capture HiC), DNA accessibility (ATAC-seq) and gene expression (RNA-seq) in FLS, B cells and T cells with genetic fine mapping of RA loci. We identified putative causal variants, enhancers, genes, and cell types for 30 - 60% of RA loci and demonstrated that FLS account for up to 24% of RA heritability. TNF stimulation of FLS altered the organization of topologically associating domains (TADs), chromatin state and the expression of putative causal genes (e.g. TNFAIP3, IFNAR1). Several putative causal genes constituted RA-relevant functional networks in FLS with roles in cellular proliferation and activation. Finally, we demonstrated that risk variants can have joint-specific effects on target gene expression in RA FLS, which may contribute to the development of the characteristic pattern of joint involvement in RA. Overall, our research provides the first direct evidence for a causal role of FLS in the genetic susceptibility for RA accounting for up to a quarter of RA heritability.