New Assessment Process Boosts Credibility of Developing-world Journals

The issue of assessing the quality of journal publishing processes is one that has vexed researchers and the publishing industry for many years, with the proliferation of online journals, open access approaches and different standards, metrics and expectations.

Many different answers have emerged or been proposed. (See, for example, Project Cupcake; Cabell’s list, reviewed here by Rick Anderson; and the Think. Check. Submit. campaign, of which INASP is a founder member).

There are strengths and weaknesses of all these ideas. However, one dangerous assumption is that any journal with some sort of gap in its approach to publishing is deliberately doing so to mislead.

Checklist and pen

INASP is an international development organization dedicated to supporting the production, sharing and use of research and knowledge in the developing world (full disclosure: I serve as INASP’s Publications and Engagement Manager). We see the reality of the harm that dubious publishing ethics have on authors who fall victim to them (this is something that we are supporting researchers to address through our AuthorAIDproject).

However, we also see the negative implications for legitimate journals if they are unfairly distrusted because they are published outside of the usual international publishing hot spots.

The worldwide scholarly publishing sector is dominated by large publishing houses in Europe and North America – the biggest five publishing companies in the world together publish around 10,000 journals (source: publishers’ own figures). In contrast, in developing countries, journals are more commonly published as stand-alone titles by universities or scholarly research associations. They are usually run by editorial boards who are doing this work on a voluntary basis. These people are subject experts, not publishing experts, so they are not necessarily aware of all the standards – particularly the newer, technical, standards — that are expected for reputable journal publishing today.

I’ve met some of the editors of journals on INASP’s Journals Online (JOL) platforms and they are passionate about their subjects and about their journals’ contribution to scholarship. But sometimes there are gaps in good practice simply because the scientists who set up the journals are not aware of global publishing practices — unsurprising really for people who spend their days in their labs, teaching students or out doing fieldwork rather than in international publishing conferences.

Some people have asked me why this matters. If Elsevier publishes a journal in a particular field of medicine, do we need a journal on a similar topic set up by a group of doctors in Bangladesh or Ethiopia?

At INASP we believe the answer is yes. As an international development organization, we see two big development reasons for supporting such scholar-led titles.

The first is about the research. Many challenges – for example, with certain diseases, cultural issues, or climate change effects – are specific to particular countries or regions and may not be priorities for journals published in the Global North. The scholars who set up a journal in this area in their country know that there is a need for this research to be disseminated. (If you are interested in some of the research being published in journals on three of the south Asian JOLs, see our research press releases).

The second issue is equity. A thriving national research and publishing sector is important for Southern countries to play an equal role in global research conversations and global decisions. As Jo Beall, Director Education and Society at the British Council and Chair of INASP’s Board of Trustees, told guests at the reception INASP hosted at the recent Going Global conference: “Without the research capability and without the use of locally produced and locally honed evidence we will not create the universal approach that is envisaged by the Sustainable Development Goals.”

However, it is not easy for Southern journals. Aside from the financial and resource challenges that come from being a stand-alone journal, particularly one in a developing country, there are the challenges of being discoverable and being trusted.

Since 1998, the Journals Online project has been addressing the first part of this challenge, providing an online platform for hosting Southern journals, first in Africa (African Journals Online – or AJOL – was the first JOL; this has been run as an independent South Africa-based entity since 2005) and then in Central America and south Asia. The JOLs are based on the Open Journal System and provide journals with many of the standard services that are expected today (for example, DOIs, iThenticate checks, and impact reporting with Kudos).

The JOL platforms have always had a basic checklist for determining whether journals can be included. However, it became clear to INASP and to our partner AJOL that there was a need to go deeper with this scrutiny, to help build trust in genuine Southern journals and also to support editors as they get to grips with today’s extensive publishing expectations. We saw the need for a new approach that would guide not only authors and readers but also editors. One that would reward existing good publishing practices while also helping those who want to improve their journals.

Journal editors from Nepal.
Journal editors on the NepJOL platform meeting in Nepal earlier this month.

The result, launching this week, is Journal Publishing Practices and Standards (JPPS). JPPS measures JOL journals against 108 criteria, which are based on extensive research into international publishing standards. These criteria measure factors such as the amount of information included on a journals’ website; the transparency and robustness of the peer-review process; and the journal’s archiving policy. But the aim here is about helping journals to improve, not about discrediting them.

Following a rigorous assessment process, journals are awarded one of six badges. These badges will inform and reassure authors and readers that journals meet international standards to a particular level. More uniquely, the framework will also provide guidance to journal editors on how they should improve their publishing processes. Editors can take advantage of various resources from INASP and AJOL to help them develop their publishing practices and then resubmit their journals for assessment after six months to a year.

Sometimes there are gaps in good practice simply because the scientists who set up the journals are not aware of global publishing practices — unsurprising really for people who spend their days in their labs, teaching students or out doing fieldwork rather than in international publishing conferences.

The process is initially focusing on the 900+ journals across the six JOL platforms (Africa, Central America, Nepal, Bangladesh, Sri Lanka and Mongolia) and we are exploring how it might be extended to other Southern journal platforms.

Last week in Nepal my colleague Sioux Cumming, INASP’s Program Manager for Journals Online, shared with journal editors their initial JPPS reports and the response was overwhelmingly positive. One comment that stood out for me was an editor who said that, to date, they had been focused on the quality of the journal content and hadn’t realized that they also needed to do so much with the quality of the publishing process.

As a former academic who is passionate about good research, I find this comment encouraging. If a journal genuinely aims to share good research then it is even more important that the people who run the journal have the support needed to ensure that their journal processes match the quality of the research that they want to disseminate.

JPPS has been almost two years in the making and there is plenty still to do – not least the fact that, even as the initial assessments continue, some journals will near the point where they can start applying for reassessment. It is an exciting project and one that will hopefully give authors more confidence in Southern journals and give those journals the credibility to play their part in global scholarship.


Leave a Reply

Your email address will not be published. Required fields are marked *