White lupine (Lupinus albus L.) is a well-known green manure crop in Hungary, but the production of seeds can be badly impacted by weeds. The sweet white lupine ‘Nelly’ was grown on acidic sandy soil, and experimental plots were treated with different herbicides. Flumioxazin (0.06 kg ha−1), pendimethalin (5 L ha−1), dimethenamid-P (1.4 L ha−1), pethoxamid (2 L ha−1), clomazone (0.2 L ha−1), metobromuron (3 L ha−1), and metribuzin (0.55 L ha−1) were applied pre-emergence (1–2 days after sowing). Imazamox was also tested and applied post-emergence (1 L ha−1) when some basal leaves were clearly distinct (BBCH 2.3). In this paper, the weed control efficiency and the phytotoxicity of herbicides applied to lupine are examined. Vegetation index datasets were collected 12 times using a manual device and 2 times using an unmanned aerial vehicle (UAV). The phytotoxicity caused by herbicides was visually assessed on several occasions throughout the breeding season. The frequency of weed occurrence per treatment was assessed. The harvested seed yields, in kg ha−1, were analyzed after the seeds were cleaned. The herbicides metribuzin and imazamox caused extensive damage to white lupine. While pendimethalin, dimethenamid-P, pethoxamid, and clomazone were outstanding in several measured indicators, the final ranking which summarizes all the variables showed that only the pethoxamid and clomazone treatments performed better than the control. Metribuzin and imazamox were highly phytotoxic to white lupine. In the future, it would be appropriate to integrate more post-emergence active substances into trials, and the pre-emergence herbicides involved in this study should be further tested.