Mobile social media applications ("apps"), such as TikTok (previously Musical.ly), have recently surfaced in news media due to harmful incidents involving young children engaging with strangers through these mobile apps. To better understand children's awareness of online stranger danger and explore their visions for technologies that can help them manage related online risks (e.g., sexual solicitations and cyberbullying), we held two participatory design sessions with 12 children (ages 8-11 years old). We found that children desired varying levels of agency, depending on the severity of the risk. In most cases, they wanted help resolving the issue themselves instead of relying on their parents to do it for them. Children also believed that social media apps should take on more responsibility in promoting online safety for children. We discuss the children's desires for agency, privacy, and automated intelligent assistance and provide novel design recommendations inspired by children.