The emergence of the nematic electronic state that breaks rotational symmetry is one of the most fascinating properties of the iron-based superconductors, and has relevance to cuprates as well. FeSe has a unique ground state in which superconductivity coexists with a nematic order without long-range magnetic ordering, providing a significant opportunity to investigate the role of nematicity in the superconducting pairing interaction. Here, to reveal how the superconducting gap evolves with nematicity, we measure the thermal conductivity and specific heat of FeSe S , in which the nematicity is suppressed by isoelectronic sulfur substitution and a nematic critical point (NCP) appears at [Formula: see text] We find that, in the whole nematic regime ([Formula: see text]), the field dependence of two quantities consistently shows two-gap behavior; one gap is small but highly anisotropic with deep minima or line nodes, and the other is larger and more isotropic. In stark contrast, in the tetragonal regime ([Formula: see text]), the larger gap becomes strongly anisotropic, demonstrating an abrupt change in the superconducting gap structure at the NCP. Near the NCP, charge fluctuations of [Formula: see text] and [Formula: see text] orbitals are enhanced equally in the tetragonal side, whereas they develop differently in the orthorhombic side. Our observation therefore directly implies that the orbital-dependent nature of the nematic fluctuations has a strong impact on the superconducting gap structure and hence on the pairing interaction.