Insulin secretion in pancreatic β-cells is regulated by cortical complexes that are enriched at the sites of adhesion to extracellular matrix facing the vasculature. Many components of these complexes, including Bassoon, RIM, ELKS and liprins, are shared with neuronal synapses. Here, we show that insulin secretion sites also contain non-neuronal proteins LL5β and KANK1, which in migrating cells organize exocytotic machinery in the vicinity of integrin-based adhesions. Depletion of LL5β or focal adhesion disassembly triggered by myosin II inhibition perturbed the clustering of secretory complexes and attenuated the first wave of insulin release. While previous analyses in vitro and in neurons suggested that secretory machinery might assemble through liquid-liquid phase separation, analysis of endogenously labeled ELKS in pancreatic islets indicated that its dynamics is inconsistent with such a scenario. Instead, fluorescence recovery after photobleaching and single molecule imaging showed that ELKS turnover is driven by binding and unbinding to low-mobility scaffolds. Both the scaffold movements and ELKS exchange were stimulated by glucose treatment. Our findings help to explain how integrin-based adhesions control spatial organization of glucose-stimulated insulin release.