The mosquito Aedes aegypti, the principal vector of dengue virus, has recently been infected experimentally with Wolbachia: intracellular bacteria that possess potential as dengue biological control agents. Wolbachia depend on their hosts for nutrients they are unable to synthesize themselves. Consequently, competition between Wolbachia and their host for resources could reduce host fitness under the competitive conditions commonly experienced by larvae of Ae. aegypti in the field, hampering the invasion of Wolbachia into natural mosquito populations. We assess the survival and development of Ae. aegypti larvae under starvation conditions when infected with each of three experimentally-generated Wolbachia strains: wMel, wMelPop and wAlbB, and compare their fitness to wild-type uninfected larvae. We find that all three Wolbachia infections reduce the survival of larvae relative to those that are uninfected, and the severity of the effect is concordant with previously characterized fitness costs to other life stages. We also investigate the ability of larvae to recover from extended food deprivation and find no effect of Wolbachia on this trait. Aedes aegypti larvae of all infection types were able to resume their development after one month of no food, pupate rapidly, emerge at a large size, and exhibit complete cytoplasmic incompatibility and maternal transmission. A lowered ability of Wolbachia-infected larvae to survive under starvation conditions will increase the threshold infection frequency required for Wolbachia to establish in highly competitive natural Ae. aegypti populations and will also reduce the speed of invasion. This study also provides insights into survival strategies of larvae when developing in stressful environments.