Despite the importance of understanding plant growth, the mechanisms underlying how plant and fruit growth declines during drought remain poorly understood. Specifically, it remains unresolved whether carbon or water factors are responsible for limiting growth as drought progresses. We examine questions regarding the relative importance of water and carbon to fruit growth depending on the water deficit level and the fruit growth stage by measuring fruit diameter, leaf photosynthesis, and a proxy of cell turgor in olive (Olea europaea). Flow cytometry was also applied to determine the fruit cell division stage. We found that photosynthesis and turgor were related to fruit growth; specifically, the relative importance of photosynthesis was higher during periods of more intense cell division, while turgor had higher relative importance in periods where cell division comes close to ceasing and fruit growth is dependent mainly on cell expansion. This pattern was found regardless of the water deficit level, although turgor and growth ceased at more similar values of leaf water potential than photosynthesis. Cell division occurred even when fruit growth seemed to stop under water deficit conditions, which likely helped fruits to grow disproportionately when trees were hydrated again, compensating for periods with low turgor. As a result, the final fruit size was not severely penalized. We conclude that carbon and water processes are able to explain fruit growth, with importance placed on the combination of cell division and expansion. However, the major limitation to growth is turgor, which adds evidence to the sink limitation hypothesis.