SUMMARYGrain chalkiness is a major concern in rice production because it impacts milling yield and cooking quality, eventually reducing market value of the rice. A gene encoding vacuolar H+ translocating pyrophosphatase (V‐PPase) is a major quantitative trait locus in indica rice, controlling grain chalkiness. Higher transcriptional activity of this gene is associated with increased chalk content. However, whether the suppression of V‐PPase could reduce chalkiness is not clear. Furthermore, natural variation in the chalkiness of japonica rice has not been linked with V‐PPase. Here, we describe promoter targeting of the japonica V‐PPase allele that led to reduced grain chalkiness and the development of more translucent grains. Disruption of a putative GATA element by clustered regularly interspaced short palindromic repeats (CRISPR)/CRISPR‐associated protein 9 suppressed V‐PPase activity, reduced grain chalkiness and impacted post‐germination growth that could be rescued by the exogenous supply of sucrose. The mature grains of the targeted lines showed a much lower percentage of large or medium chalk. Interestingly, the targeted lines developed a significantly lower chalk under heat stress, a major inducer of grain chalk. Metabolomic analysis showed that pathways related to starch and sugar metabolism were affected in the developing grains of the targeted lines that correlated with higher inorganic pyrophosphate and starch contents and upregulation of starch biosynthesis genes. In summary, we show a biotechnology approach of reducing grain chalkiness in rice by downregulating the transcriptional activity of V‐PPase that presumably leads to altered metabolic rates, including starch biosynthesis, resulting in more compact packing of starch granules and formation of translucent rice grains.