Research-impact nexuses and thinking beyond the framework

The author gave a presentation on the topic of this blog in workshop 2: Rethinking Policy Impact: types of knowledge.

I describe these nexuses through several features, which go beyond the inherent ambiguity and constructedness of impact as an object of assessment. They are domains where it is difficult to make a distinction or a demarcation between research itself, practice, and research impacts; where there is a potential conflict between the aims and the values that underpin specific modes of research, and the more mainstream or top-down understandings of reach and significance that underpin the methods, indicators and metrics for assessing impact; and also where there is a lot of uncertainty and ambiguity around the distinction between pathways to impact and actual impacts.

These hard-to-assess domains are relational, dynamic and synergetic; they have been and continue to be very difficult, if not impossible, to capture within the technical definitions and institutional frames that we have for assessing research impact, particularly as part of performance-based funding exercises. Such research-impact nexuses include: the critical, emancipatory, and subversive research-impact nexus; the discursive and conceptual research-impact nexus; the collective, reciprocal and deeply collaborative research-impact nexus; the creative, craft and design-based research-impact nexus; and the professionally-oriented and practice-based research-impact nexus. Fitting them into assessment templates that expect separate accounts of research, pathways to impact, and impact, is often an exercise in artificial and instrumental re-storying that may be at odds with the understandings and values of those involved in this work.

For example, critical, emancipatory, and subversive research may lead to changes that are ‘negative’ or pre-emptive, or counter-hegemonic. The significance of such changes and the strength of the evidence for them may be mis-recognised in assessment systems that are best calibrated for positive and linear theories of change. Case studies submitted to the REF may (although sporadically) reference such impacts, but they also acknowledge openly how difficult it is to provide evidence of a ‘distinct and material contribution’ from specific research to impacts of this kind without mis-representing the intention and value commitments behind the work; as a result, they tend to focus instead on participation in, and volume of, “pathway” activities. There is a positive bias in selecting many of the latter, of course – as, arguably, evidencing preventative action may be at least as, if not more, difficult than evidencing catalytic influence or engagement in critical debate. Further, critical research may grow from philosophical and theoretical positions that are fundamentally at odds with hegemonic narratives and with the political philosophies underpinning current policy making. Such research may seek radical change rather than incremental adjustments to current frames; and as such, working with the grain of policy rather than against it may be seen as a compromise too far. In attempting to report such impacts for assessment purposes, there is therefore always a risk of (selective) co-option of critique as a way of washing over inequalities and injustices that remain essentially unchallenged.

As a second example, the contributions of research to discursive and conceptual change may develop over very long periods of time and through processes of percolation and osmosis, rather than direct, targeted action. In such cases, the indicators of change may be on such extended timeframes that describing pathways and beneficiaries (or ‘reach’, in UK REF parlance) may become meaningless. In addition, slow-developing changes in the language that is used in policy, professional, media and public domains to redefine and reframe specific social phenomena may pass unnoticed: we find ourselves talking about bureaucracies, capabilities, language games, discourses, structural inequalities, intersectionality, ecosystems, metacognition, mindsets, communities of practice and so on without necessarily being able to trace the scholarly sources of such concepts. Notably, such sources that remain unacknowledged in policy and practical change may involve multiple scholars and multiple studies over time, to which individuals may have been exposed repeatedly and indirectly (for example through training, conferences and informal interactions).

A final example is that of collective, participatory, reciprocal and deeply collaborative research-impact nexuses. When change arises from collective momentum and distributed exchanges involving way more knowledge agents than academic researchers (and from within and beyond the HE sector) – rather than from targeted efforts by individual researchers or teams – contribution is hard to define, intellectual property may be difficult to apportion, and claims that trace the relationships between specific research efforts and benefits are difficult to evidence. There is thus no single ‘hero’ to be celebrated: each contribution is incremental, the change arises from collaboration, replication, critique and revision, rather than from singular breakthrough and individual originality. As a result, such contributions may be systematically underreported. These are not ‘just’ technical difficulties. Indeed, the right for university-based researchers to claim impact from participatory and co-produced research may be questionable and important impacts with community roots may remain unreported for ethical reasons. Notably, assessment systems are typically not well calibrated to avoid rewarding extractive, exploitative and inequitable research and impacts.

There are of course other examples I could bring here to illustrate the notion of the deeply synergetic, dynamic, and relational research-impact nexuses that may best describe large areas of work not only in the social sciences, humanities, and arts, but also in STEM research. These are all areas that resist framing for assessment, and as a result have traditionally either been misrecognised and under-represented in impact assessment systems, or have been instrumentalised and simplified to make them intelligible to assessment procedures. Both the way in which such nexuses resist rigid frames of assessment and the consequences of their misrecognition matter beyond the logic of assessment exercises. They have social and material implications for the interpretations of research and impact that shape working conditions, institutional missions and career and performance criteria in higher education (and more widely in research environments).   

The notion of research-impact nexus introduced here may thus be relevant much more broadly, for example as a lens to inspire less instrumental conceptualisations and approaches to impact; or as a way of further articulating the connections and flows between research practice and policy. As the systems and metrics for assessing research are being reconsidered, with ongoing reviews in the UK, in the European Union, and elsewhere, it is important to reflect, not on ways to assimilate and domesticate hard-to-assess research-impact nexuses, but on how to recognise them as such and to draw on them to develop and sustain more responsible, caring and diverse cultures of research and impact. 

Note: this piece is based on Oancea, A. (accepted for publication) ‘ Beyond the frame: hard-to-assess research-impact nexuses in the social sciences and the humanities’, in: Ochsner, M. and Bulaitis, Z. (accepted for publication) Accountability in Academic Life: European Perspectives on Societal Impact Evaluation. Edward Elgar.