The idea that a longer duration of untreated psychosis (DUP) leads to poorer outcomes has contributed to extensive changes in mental health ser­vices worldwide and has attracted considerable research interest over the past 30 years. However, the strength of the evidence underlying this notion is unclear. To address this issue, we conducted an umbrella review of available meta‐analyses and performed a random‐effects meta‐analysis of primary studies. MEDLINE, Web of Science, PsycINFO and EMBASE were searched from inception to September 3, 2020 to identify relevant meta‐analyses of studies including patients with schizophrenia spectrum disorders, first‐episode psychosis, or affective and non‐affective psychosis. Thirteen meta‐analyses were included, corresponding to 129 individual studies with a total sample size of 25,657 patients. We detected potential violations of statistical assumptions in some of these meta‐analyses. We therefore conducted a new random‐effects meta‐analysis of primary studies. The association between DUP and each outcome was graded according to a standardized classification into convincing, highly suggestive, suggestive, weak, or non‐significant. At first presentation, there was suggestive evidence for a relationship between longer DUP and more severe negative symptoms (beta=–0.07, p=3.6×10–5) and higher chance of previous self‐harm (odds ratio, OR=1.89, p=1.1×10–5). At follow‐up, there was highly suggestive evidence for a relationship between longer DUP and more severe positive symptoms (beta=–0.16, p=4.5×10–8), more severe negative symptoms (beta=–0.11, p=3.5×10–10) and lower chance of remission (OR=2.16, p=3.0×10–10), and suggestive evidence for a relationship between longer DUP and poorer overall functioning (beta=–0.11, p=2.2×10–6) and more severe global psychopathology (beta=–0.16, p=4.7×10–6). Results were unchanged when analysis was restricted to prospective studies. These effect sizes are clinically meaningful, with a DUP of four weeks predicting >20% more severe symptoms at follow‐up relative to a DUP of one week. We conclude that DUP is an important prognostic factor at first presentation and predicts clinically relevant outcomes over the course of illness. We discuss conceptual issues in DUP research and methodological limitations of current evidence, and provide recommendations for future research.