The evaluation of the Austrian START programme: an impact analysis of a research funding programme using a multi-method approach

Seus, Sarah; Bührer, Susanne

doi:10.1007/s10961-017-9606-8

Cited by 4 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Again, quantitative and qualitative data were used, relying on different sources (Seus & Bührer, 2017). Some prospective impact assessment practices include general equilibri um models, data links, scientometrics, indicators based on research and in combination with econometric analyses, and case studies.…”

Section: Research To Impact Pathwaysmentioning

confidence: 99%

II. Research Impact Assessment

Vutsova¹,

Yalamov²,

Arabadzhieva³

2023

In Search of Excellent Research Assessment

View full text Add to dashboard Cite

The routes of impact assessment could be traced back to the cross-impact assessment proposed by Gordon and Helmer in 1966 (Gordon, 1994), the environmental impact statements since 1970 (National Environmental Policy Act), and technology assessments (Coates, 1971). All these methods of futures studies aimed to develop plausible scenarios and strategies to cope with the growing uncertainty.The impact assessment is an ongoing process of monitoring and analys ing the social, economic, and ecological changes which occur as a result of the implementation of a given activity. The objectives of an impact assessment are usually aligned with the functions of a given organisation, and in specific cases they are independent of regulatory factors (ecological assessments are an exception, for example). The impact assessments usually look beyond the standard horizon of planning of the activities.An impact assessment usually surpasses the boundaries of the 'gross' results and impacts foreseen in a given policy, programme, project, or initiative. Sometimes impact assessments are conducted within the larger context of foresight studies, which are trying to back-cast what should be done in order to reach a desirable future or to avoid undesirable one. The time-horizons are always extending and the overall uncertainty grows, thus calling for foresight-based capacity to react to uncertainty shocks. The impact assessment could be considered also as a part of broader agenda of 'evidence-based policy making'. It also can be used to measure programmes implementation alternatives and their innovativeness.In general, 'impact assessment' deals with the effects of proposed and/or planned actions (Porter & Rossini, 2019). The International Association for Impact Assessment (Fargo, North Dakota, USA) accepts that the impact as sessment is the process of identifying the future consequences of a current or proposed action through which social justice and quality of the environ ment are achieved to a certain extent. According to this association, the impact assessment is one of the approaches for analysing policies and pro grammes, and is also complemented by a technological assessment and a risk assessment (International Association for Impact Assessment). Impact II.

show abstract

Section: Research To Impact Pathwaysmentioning

confidence: 99%

II. Research Impact Assessment

Vutsova¹,

Yalamov²,

Arabadzhieva³

2023

In Search of Excellent Research Assessment

View full text Add to dashboard Cite

show abstract

“…Small to moderate effects of funding on performance were found in biology, but not in medicine. Seus and Bührer (2017) studied the effects of the Austrian Science Fund' START programme on grantees' research performance and career development. Publication productivity of the grantees increased during and after the grant period and was higher than that of the control group.…”

Section: Previous Researchmentioning

confidence: 99%

Research performance and scholarly communication profile of competitive research funding: the case of Academy of Finland

Pölönen

Auranen

2022

Scientometrics

View full text Add to dashboard Cite

The Academy of Finland (AKA), Finland’s major public research funding agency, uses a Web of Science (WoS) based bibliometric indicator to assess the performance of research it has funded. We use an alternative methodology to compare (1) the research performance and (2) the scholarly communication profile of AKA-funded research to the Finnish universities’ entire output across the major fields of arts and sciences. Our data consists of 142,742 publications (years 2015–2018) registered in the national information service, which integrates Current Research Information System (CRIS) data of 13 Finnish universities. Research performance is analyzed using the Finnish community-curated expert-based rating of publication channels (so-called JUFO). Our results show that compared to the Finnish universities’ entire output a larger share of AKA-funded research is published in leading JUFO rated journals and book publishers. JUFO and WoS-based indicators produced consonant results regarding the performance of AKA-funded research. Analysis of publication profiles shows that AKA-funded research is more focused than the universities’ output on using peer-reviewed publications, articles published in journals, English language, foreign publishers and open access publishing. We conclude that the CRIS-based publication data can support multidimensional assessments of research performance and scholarly communication profiles, potentially also in other countries and institutions. CRIS development and maintenance require multi-stakeholder commitment, resources and incentives to ensure data quality and coverage. To fully recognize diverse open science practices and to enable international comparisons, CRISs need further development and integration as data sources.

show abstract

How to assess the impact of fellowships on academic careers? Latent transition analyses for funding programmes of the Alexander von Humboldt Foundation

Mutz,

Daniel

2024

Qual Quant

View full text Add to dashboard Cite

Although fellowship programmes continue to be seen as an effective means of advancing young researchers' academic careers, the impact of fellowship programmes on fellows' career development is still unclear. The central question of this article concerns the evaluation of fellowship programmes: What methodological challenges does the evaluation of fellowship programmes pose with regard to career development, and how these can be addressed in the context of evaluations? Specifically, there are three key methodological challenges facing research evaluation in the context of career development, which can be described by the terms 'impact', 'validity and fairness', and 'tailored programmes'. A career is understood as a transition between positions over time; career stages can be understood as latent categorical variables, i.e. types of career stages (temporary, full-time). Transition is modelled statistically using latent transition analyses within a person-centred approach. Covariates, such as funding programmes, can impact both the initial configurations (i.e. the frequency of fellows in different career stages) and the transition itself. A funding programme is fair if all fellows, regardless of their characteristics (gender, career stage, cohort), have the same chances of success. Different types of fellows with different career trajectories indicate heterogeneous subpopulations that require tailoring of funding programmes. The approach is illustrated with data on the career development of 1418 fellows from three Alexander von Humboldt Foundation programmes. The majority of fellows benefit in their academic careers from the funding, but the null hypothesis of no specific effects (e.g. programmes, age, gender) could not be rejected (endogenous and homogeneous trajectories).

show abstract

The evaluation of the Austrian START programme: an impact analysis of a research funding programme using a multi-method approach

Cited by 4 publications

References 19 publications

II. Research Impact Assessment

II. Research Impact Assessment

Research performance and scholarly communication profile of competitive research funding: the case of Academy of Finland

How to assess the impact of fellowships on academic careers? Latent transition analyses for funding programmes of the Alexander von Humboldt Foundation

Contact Info

Product

Resources

About