The rise of antimicrobial resistance (AMR) is one of the greatest public health challenges, already causing up to 1.2 million deaths annually and rising. Current gold-standard antimicrobial susceptibility tests (ASTs) are low-throughput and can take up to 48 hours, with implications for patient care. We present advances towards a novel, rapid AST, based on the deep-learning of single-cell specific phenotypes directly associated with antimicrobial susceptibility in Escherichia coli. Our models can reliably (80% single-cell accuracy) classify untreated and treated susceptible cells, across a range of antibiotics and phenotypes - including phenotypes not visually distinct to a trained, human observer. Applying models trained on lab-reference susceptible strains to clinical isolates of E. coli treated with ciprofloxacin, we demonstrate our models reveal significant (p<0.001) differences between resistant and susceptible populations, around a fixed treatment level. Conversely, deploying on cells treated with a range of ciprofloxacin concentrations, we show single-cell phenotyping has the potential to provide equivalent information to a 24-hour growth AST assay, but in as little as 30 minutes.