IMPORTANCE Evidence supports use of teleconsultation for improving patient access to dermatology. However, little is known about the quality of rapidly expanding direct-to-consumer (DTC) telemedicine websites and smartphone apps diagnosing and treating skin disease. OBJECTIVE To assess the performance of DTC teledermatology services. DESIGN AND PARTICIPANTS Simulated patients submitted a series of structured dermatologic cases with photographs, including neoplastic, inflammatory, and infectious conditions, using regional and national DTC telemedicine websites and smartphone apps offering services to California residents. MAIN OUTCOMES AND MEASURES Choice of clinician, transparency of credentials, clinician location, demographic and medical data requested, diagnoses given, treatments recommended or prescribed, adverse effects discussed, care coordination. RESULTS We received responses for 62 clinical encounters from 16 DTC telemedicine websites from February 4 to March 11, 2016. None asked for identification or raised concerns about pseudonym use or falsified photographs. During most encounters (42 [68%]), patients were assigned a clinician without any choice. Only 16 (26%) disclosed information about clinician licensure, and some used internationally based physicians without California licenses. Few collected the name of an existing primary care physician (14 [23%]) or offered to send records (6 [10%]). A diagnosis or likely diagnosis was proffered in 48 encounters (77%). Prescription medications were ordered in 31 of 48 diagnosed cases (65%), and relevant adverse effects or pregnancy risks were disclosed in a minority (10 of 31 [32%] and 6 of 14 [43%], respectively). Websites made several correct diagnoses in clinical scenarios where photographs alone were adequate, but when basic additional history elements (eg, fever, hypertrichosis, oligomenorrhea) were important, they regularly failed to ask simple relevant questions and diagnostic performance was poor. Major diagnoses were repeatedly missed, including secondary syphilis, eczema herpeticum, gram-negative folliculitis, and polycystic ovarian syndrome. Regardless of the diagnoses given, treatments prescribed were sometimes at odds with existing guidelines. CONCLUSIONS AND RELEVANCE Telemedicine has potential to expand access to high-value health care. Our findings, however, raise concerns about the quality of skin disease diagnosis and treatment provided by many DTC telemedicine websites. Ongoing expansion of health plan coverage of these services may be premature. Until improvements are made, patients risk using health care services that lack transparency, choice, thoroughness, diagnostic and therapeutic quality, and care coordination. We offer several suggestions to improve the quality of DTC telemedicine websites and apps and avoid further growth of fragmented, low-quality care.