Smart toys have captured an increasing share of the toy market, and are growing ubiquitous in households with children. Smart toys are a subset of Internet of Things (IoT) devices, containing sensors, actuators, and/or artificial intelligence capabilities. They frequently have internet connectivity, directly or indirectly through companion apps, and collect information about their users and environments. Recent studies have found security flaws in many smart toys that have led to serious privacy leaks, or allowed tracking a child's physical location. Some well-publicized discoveries of this nature have prompted actions from governments around the world to ban some of these toys. Compared to other IoT devices, smart toys pose unique risks because of their easily-vulnerable user base, and our work is intended to define these risks and assess a subset of toys against them. We provide a classification of threats specific to smart toys in order to unite and complement existing adhoc analyses, and help comprehensive evaluation of other smart toys. Our vulnerability taxonomy addresses the potential security and privacy flaws that can lead to leakage of private information or allow an adversary to control the toy to lure, harm, or distress a child. Using this taxonomy, we perform a thorough experimental analysis of eleven smart toys and their companion apps. Our systematic analysis has uncovered that several current toys still expose children to multiple threats for attackers with physical, nearby, or remote access to the toy.