Defining impact for a world renowned scientific journal

Customer

A world renowned academic science journal.

Problem

The journal wanted to understand how to measure the breadth of its impact and influence beyond the science community. Conventional citation methods used in academia (such as impact factor and H-index) can be biased through self-citation, backward looking, and are not always good predictors of future impact. The organisation worked with Faculty to gain a holistic understanding of its impact by using artificial intelligence to determine new ways to measure impact and influence.

Solution

We developed a strategic framework that helped define impact in the journal’s own context as well as the industry as a whole. Firstly, we worked with the journal and other entities in the publishing group to create a clear picture and understand the complexities of their current measurement systems. We were then able to pinpoint why the existing measurement landscape wasn’t sufficient.

We went on to perform a benchmarking exercise and literature analysis to understand the current ways impact in journalism is measured in the wider market to inform new ways this could be measured. A wide variety of techniques were included to solve the problem of measuring impact including clustering, natural language processing, topic analysis, and graph theory. With these techniques we were able to move away from citation based measurements towards other features of journal articles that might indicate impact such as a text body, publishing statistics and authorship / collaboration. 

Through this exercise, we were able to understand where the gaps are in impact measurement, and developed nine potential impact measurement approaches. The organisation then selected three to investigate further to determine their feasibility.

Impact

The journal is now building the approaches we defined into its overall publishing group strategy to help redefine how impact is measured in the industry as a whole.