BackgroundThere is a need to develop innovations that can help bridge the gap between research and policy. Web CIPHER is an online tool designed to help policymakers better engage with research in order to increase its use in health policymaking. The aim of the present study was to test interventions in order to increase policymakers’ usage of Web CIPHER. Namely, the impact of posting articles and blogs on topics relevant to the missions and scope of selected policy agencies in the Web CIPHER community.MethodsFive policy agencies were targeted for the intervention. Web CIPHER usage data was gathered over a 30-month period using Google Analytics. Time series analysis was used to evaluate whether publication of tailored articles and blogs led to significant changes in usage for all Web CIPHER members from policy agencies, including those from the five target agencies. We further evaluated whether these users showed greater increases in usage following publication of articles and blogs directly targeted at their agency, and if these effects were moderated by the blog author.ResultsWeb CIPHER usage gradually increased over time and was significantly predicted by the number of articles but not blogs that were posted throughout the study period. Publication of articles on sexual and reproductive health was followed by sustained increases in usage among all users, including users from the policy agency that targets this area. This effect of topic relevance did not occur for the four remaining target agencies. Finally, page views were higher for articles targeted at one’s agency compared to other agencies. This effect also occurred for blogs, particularly when the author was internal to one’s agency.ConclusionThe findings suggest that Web CIPHER usage in general was motivated by general interest, engagement and appeal, as opposed to the agency specificity of content and work relevance. Blogs in and of themselves may not be effective at promoting usage. Thus, in order to increase policymakers’ engagement with research through similar online platforms, a potentially effective approach would be to post abundant, frequently updated, engaging, interesting and widely appealing content irrespective of form.Electronic supplementary materialThe online version of this article (doi:10.1186/s12961-016-0157-5) contains supplementary material, which is available to authorized users.