From Research Output to Research Influence: The Metrics Universities Should Watch

Influence begins where counting ends - but it still needs disciplined evidence.

Once a university accepts that publication volume cannot describe research strength on its own, a second problem appears: influence is harder to measure than output.

A paper can be counted the day it enters a database. Influence may emerge slowly, through citations, adoption, replication, professional practice, public policy, technology, culture or the creation of new research questions. Some influence is visible numerically. Some is best demonstrated through documented cases.

The challenge for institutions is to avoid exchanging one simplistic metric for another. Research influence is multidimensional, and the measurement system should be too.

Start With Citations - Then Read Them Properly

Citation data remain useful because they show that a publication has entered subsequent scholarly conversation. But raw citation counts are highly sensitive to discipline, publication year, document type, database coverage and the size of the field.

That is why field-normalised measures are generally more informative than simple totals for cross-disciplinary comparison. They ask whether a work is cited more or less than would typically be expected for comparable research in the same field and period.

Even then, interpretation matters. A citation indicates that a work was referenced; it does not prove that the citing author agreed with it or that the research created beneficial impact. Citation metrics are evidence of scholarly attention, not a complete account of quality or consequence.

Look Beyond the Journal Literature

Research often travels through channels that bibliometric databases do not fully capture.

A public-policy team may use findings in guidance or regulation. A hospital may change a protocol. An engineering partner may alter a design standard. A software tool or dataset may be reused widely. A local authority may adopt a planning model. A cultural project may change public interpretation of an issue.

The UK's REF provides a useful illustration of this broader logic: it defines research impact as an effect, change or benefit beyond academia and assesses impact through evidenced case studies. The framework also recognises the reduction or prevention of harm, risk, cost or other negative effects as potential impact.

For universities, the lesson is not to copy one national system. It is to recognise that influence outside academia often requires documentary evidence and narrative explanation alongside metrics.

Five Families of Indicators Worth Watching

A useful research-influence dashboard can be organised around five families of evidence.

First, scholarly influence: field-normalised citations, highly cited outputs where appropriate, review articles, replication and sustained scholarly use. Second, network influence: the quality and continuity of research collaborations, co-authorship patterns and participation in major disciplinary or international consortia. Third, translation and adoption: evidence that findings, tools, datasets or methods have been used in policy, professional practice, industry, healthcare, technology or public services.

Fourth, innovation pathways: patents, licensing, standards contributions, spin-outs, open-source deployment or other forms of application where these are relevant to the discipline. Fifth, public and societal contribution: credible evidence of changes in public understanding, cultural practice, community outcomes or other benefits that may not generate conventional commercial or citation signals.

The point is not to maximise every indicator. It is to understand which pathways matter for the institution's research portfolio.

Relationships Can Be Evidence, But They Need Interpretation

Repeat collaboration can be a useful signal of trust and value, especially when partners choose to return for further projects. Long-term co-research relationships, recurring commissioned work and multi-year participation in international networks may show that an institution has become a dependable research partner.

But collaboration counts should not be treated as automatic evidence of influence. A large number of co-authored papers can arise from very different levels of contribution. A partnership can be prestigious but shallow. Network data should therefore be read together with evidence about role, continuity, outputs and outcomes.

This distinction is especially important for universities building global research strategies: visibility inside a network is not the same as being intellectually central to it.

The Time Horizon Matters

Influence rarely moves at the speed of annual reporting. Some research is applied quickly; other work takes years to accumulate recognition or reach practice.

Institutions therefore need leading and lagging indicators. Early signals might include external invitations, new collaborations, replication, data reuse or uptake in technical discussion. Longer-term signals might include policy change, clinical adoption, standards, commercialisation, enduring citation influence or sustained societal benefit.

A research office that expects every project to demonstrate mature impact within the same short window risks favouring work with fast, visible pathways over work whose contribution is slower but potentially substantial.

HERA Perspective: Build an Evidence Portfolio, Not a Magic Number

The search for one perfect metric of research influence is likely to end where the search for one perfect metric of research quality did: with an indicator carrying more meaning than it can support.

A stronger approach is an evidence portfolio. Quantitative indicators reveal patterns at scale; qualitative review explains context; impact cases show how research travelled; longitudinal tracking shows whether influence endured.

The most useful research dashboard is therefore not the one with the most numbers. It is the one that enables leadership to answer a better question: where, how and for whom is our research making a demonstrable difference?