Introduction: Pancreatic neuroendocrine neoplasms (PNENs) present with a fibrotic stroma that constitutes the tumor microenvironment (TME). The role played by stromal fibroblasts in the growth of PNENs and their sensitivity to the mTOR inhibitor RAD001 has not yet been established. Methods: We investigated reciprocal interactions between (1) human PNEN cell lines (BON-1/QGP-1) or primary cultures of human ileal neuroendocrine neoplasm (iNEN) or PNEN and (2) human fibroblast cell lines (HPF/HFL-1). Proliferation was assessed in transwell (tw) co-culture or in the presence of serum-free conditioned media (cm), with and without RAD001. Colony formation and migration of BON-1/QGP-1 were evaluated upon incubation with HPFcm. Results: Proliferation of BON-1 and QGP-1 increased in the presence of HFL-1cm, HPFcm, HFL-1tw and HPFtw (BON-1: +46–70% and QGP-1: +42–55%, p < 0.001 vs. controls) and HPFcm significantly increased the number of BON-1 or QGP-1 colonies (p < 0.05). This stimulatory effect was reversed in the presence of RAD001. Likewise, proliferation of human iNEN and PNEN primary cultures increased in the presence of HFL-1 or HPF. Reciprocally, BON-1cm and BONtw stimulated the proliferation of HPF (+90 ± 61% and +55 ± 47%, respectively, p < 0.001 vs. controls), an effect less pronounced with QGP-1cm or QGPtw (+19 to +27%, p < 0.05 vs. controls). Finally, a higher migration potential for BON-1 and QGP-1 was found in the presence of HPFcm (p < 0.001 vs. controls). Conclusions: Fibroblasts in the TME of PNENs represent a target of interest, the stimulatory effect of which over PNENs is mitigated by the mTOR inhibitor everolimus.