Quantitative assessments of tumor burden and modeling of longitudinal growth could improve phase II oncology trials. To identify obstacles to wider use of quantitative measures we obtained recorded linear tumor measurements from three published lung cancer trials. Model‐based parameters of tumor burden change were estimated and compared with similarly sized samples from separate trials. Time‐to‐tumor growth (TTG) was computed from measurements recorded on case report forms and a second radiologist blinded to the form data. Response Evaluation Criteria in Solid Tumors (RECIST)‐based progression‐free survival (PFS) measures were perfectly concordant between the original forms data and the blinded radiologist re‐evaluation (intraclass correlation coefficient = 1), but these routine interrater differences in the identification and measurement of target lesions were associated with an average 18‐week delay (range, −20 to 55 weeks) in TTG (intraclass correlation coefficient = 0.32). To exploit computational metrics for improving statistical power in small clinical trials will require increased precision of tumor burden assessments.