(1) Background: Pelvic organ prolapse (POP) can be managed using a vaginal pessary. However, the decision-making process whereby health professionals choose the right pessary is unclear. The objective of this study was to focus on the experience of experts in pessary use and to propose an algorithm. (2) Methods: A prospective study, based on face-to-face semi-directive interviews and group discussions, was conducted on a multidisciplinary panel of professional experts specialized in pessary prescriptions. A consensual algorithm was established, and its accuracy was assessed by expert and non-expert panels. The Consolidated Criteria for Reporting Qualitative Studies (COREQ) were used. (3) Results: 17 semi-directive interviews were conducted. The parameters involved in the decision-making process regarding the choice of vaginal pessaries were: desire for self-management (65%), associated urinary stress incontinence (47%), POP type (41%), and POP stage (29%). The algorithm was developed step by step (4 iterations) using the Delphi technique. Most of the expert panel (76%) rated the relevance of the algorithm as 7 or more out of 10 on a visual analog scale according to their own experience (reference activity). Finally, most (81%) of the non-expert panel (n = 230) rated the usefulness of this algorithm as 7 or more out of 10 on a visual analog scale. (4) Conclusions: This study provides an expert panel-based algorithm that may help in the prescription of pessaries for POP.