Double Field Theory (DFT) is an attempt to make the O(d, d) T-duality symmetry of string theory manifest, already before reducing on a d-torus. It is known that supergravity can be formulated in an O(D, D) covariant way, and remarkably this remains true to the first order in α′. We set up a systematic way to analyze O(D, D) invariants, working order by order in fields, which we carry out up to order α′3. At order α′ we recover the known Riemann squared invariant, while at order α′2 we find no independent invariant. This is compatible with the α′ expansion in string theory. However, at order α′3 we show that there is again no O(D, D) invariant, in contradiction to the fact that all string theories have quartic Riemann terms with coefficient proportional to ζ (3). We conclude that DFT and similar frameworks cannot capture the full α′ expansion in string theory.