Usually, Web-based graphical user interfaces (GUIs) are not specifically tailored for different devices with touchscreens, such as smartphones, where interaction is affected mainly by screen size. There is little scientific evidence on the benefits of tailoring in general, and in particular on the conditions where scrolling is good or bad. Therefore, we conducted a user study in which we experimentally evaluated a GUI tailored for a smartphone and another non-tailored one. The tailoring in this case only rearranges widgets in a way that the width of the device screen is sufficient but vertical scrolling may be necessary. Each participant performed the same task with these two different layouts. We collected quantitative data through measuring task completion time and error rates, as well as qualitative data through subjective questionnaires and interviews. The main result is that tailoring a GUI for a smartphone is important, since task performance time was significantly shorter when using a tailored GUI requiring only vertical scrolling as compared to a non-tailored one. This preference was also reflected in the subjective opinions of the users.