Voting Advice Applications (VAAs) provide nonpartisan and interactive information during election campaigns, and match the policy demand of users with the political supply of parties. But do VAAs influence citizens’ electoral decisions? Do they help the undecided to form a political preference, and lead the decided to change their vote choice? This article reports a randomized field experiment to evaluate the effect of the Vote Compass, a VAA used in Quebec, on users’ electoral preferences. Given Quebec's multidimensional political space, VAAs have the potential to assist citizens in making a complex electoral decision. The results show that the VAA's users are more likely to form an electoral preference, but this is only the case in the short term among 30‐year‐olds, the higher educated, and more politically interested users. At the same time, using a VAA mostly impacts party preferences of voters in the short term: the politically uninterested and older users are more likely to change their party preferences, whereas 30‐year‐olds and politically interested users are less likely to change their initial party preference. This study provides evidence that the VAA cognitively engaged users, but not that it leads to vote switching on Election Day.