Measuring your Research Impact
Bibliometric data provide internationally recognised quantitative indicators for measuring and tracking the impact of scholarly publications. The data is used (alongside qualitative inputs) to measure and report the impact and influence of the research output of an individual researcher or the combined output of a department, centre, institution or country. This quantitative analysis of research publications is based on citation data.
Support and Contacts
The Library offers advice, support and training on the tools available to gather research performance data. Please contact your Subject Librarian to discuss your needs. For example we can help you answer questions such as:
- How many citations have my publications received?
- How do I calculate my h:index?
- How do I find out which journals are in the top quartile for my field?
- What is the expected citation rate for papers in my subject field?
Below you will find information on …….
Metrics range from simple publication or citation counts to mathematical formulae which take into account both the output and impact of a researcher’s work.
Publication count - the number of publications produced by an individual, unit or institution is the most basic metric used to measure productivity or output.
Citation count - the number of citations a paper (or group of papers) receives is used to measure impact or influence.
H-index - the H-index takes into account productivity (paper counts) and impact (citations). If an individual has a h-index of 7 this means that 7 of their papers have been cited at least 7 times.
The main tools used for bibliometric analysis are:
Web of Science - Subscription based service. Provides access to over 12,000 peer reviewed journals across the sciences, social sciences, arts, and humanities in addition to proceedings of international conferences, symposia, seminars, colloquia, workshops, and conventions.
Scopus - Subscription based abstract and citation database of peer-reviewed literature from 22,000+ journals from 5,000 publishers in the fields of science, technology, medicine, social sciences, and arts and humanities. Also indexes books, conference papers and patents.
Google Scholar - Search across scholarly content from journal articles, theses, books, abstracts and court opinions, from academic publishers, professional societies, online repositories, universities and other web sites
There are also some specialised tools for certain disciplines e.g. MathSciNet (Maths), ACM Digital Library (Computing).
Things you should know…The results you get will depend on the tool used. Publication, citation counts and therefore other metrics will vary as each database (tool) indexes different content (sources). You should be aware of the pros and cons of each.
For more information and to learn how to use these tools to calculate publication counts, citations and h-index have a look at the MyRI Tutorial.
Module 2 - Tracking your Research Impact
The ranking of a journal (journal impact) is often taken into consideration by academics and researchers when choosing which journal to publish in. The metrics used to measure journal impact vary depending on the tool used. The following are some of the main metrics used and how you can access them.
Journal Impact Factor (JIF)
The journal impact factor is the average number of citations received in a year by articles published in a journal in the previous two years. e.g. a journal’s JIF for the year 2013 is calculated as follows:
JIF is available from Journal Citation Reports (JCR) which is produced by Clarivate Analytics (formerly Thomson Reuters) and based on data from the Web of Science. JCR is the original journal ranking tool, first developed in the 1950s. Using JCR you can search for individual journals or compare groups of journals by subject category. In addition, JCR provides a range of other metrics for e.g. Impact Factor without Journal Self Cites, 5 Year Impact Factor
Produced by Elsevier, CiteScore is a set of journal metrics based on citation data from the Scopus database. A journal's CiteScore represents the average number of citations received in a given year (e.g. 2015), by papers published in the previous three years (e.g. 2012, 2013 and 2014).
SNIP Source Normalized Impact per Paper
SNIP measures contextual citation impact by weighting citations based on the total number of citations in a subject field. The impact of a single citation is given higher value in subject areas where citations are less likely, and vice versa. This is designed to allow comparison across research areas, not possible with most metrics
Scimago Journal Ranking (SJR)
The SJR is much like the Journal Impact Factor in principle. However it goes a step further by mimicking the Google PageRank algorithm. As such it assigns higher value/weight to citations from more prestigious journals. The SJR covers a three year citation window e.g. a journal’s SJR for 2010:
Things you should know…No one tool covers every journal. Citation patterns vary significantly between disciplines therefore the JIF cannot be used to compare journals across different subject areas. The impact of a specific article cannot be measured by journal impact metrics.
For more information on journal ranking tools and the issues and limitations of using such measures see MyRI Tutorial
Contact your Subject Librarian if you need advice and support on gathering research impact data for your school or centre. The Library can assist schools and centres undertaking reviews or evaluations to gather data such as
- Overall number of publications/citations and h-index score
- Average citations per document (citation impact)
- How your school or centre is performing e.g. below or above the world average in a particular subject category
- The percentage of documents appearing in the top 1%, 10%, 25% of journals in a selected category
- How your school or centre compares to selected peer institutions in one or more subject categories
- Review the impact of existing research collaborations (national and international)
- Identify potential national and international collaborations
Note: The completeness of the above data is limited to the extent of the coverage within the Web of Science and Scopus databases.
Altmetrics, short for “Alternative Metrics” provide researchers with additional data/metrics to help demonstrate a publication’s reach and importance. Altmetrics are “used to quantify how individual articles are being discussed, shared and used” (Sparc Primer).
What do Altmetrics capture?
- Usage (e.g. number of article downloads or page views)
- Captures (e.g. number of times an article is shared on Medeley)
- Mentions (e.g. in web news stories, blogs or comments added to publisher sites)
- Social Media (e.g. number of times an article is shared on Twitter or Facebook,)
How do I retrieve alternative metrics?
- At the article level from certain publisher platforms e.g. Public Library of Science (PLOS), Institute of Physics Publishing
- From the Scopus Citation Database (available to DCU staff and students)
When you search for an article on Scopus, you can now see if, and how many times, it’s been tweeted, mentioned in news sources, or saved on Mendeley etc. Scopus uses ‘Altmetric’ from altmetric.com to deliver this data.
- You can also download the free “Altmetric it” bookmarlet (from Altmetric.com) by dragging the ‘Altmetric’ icon to your browser’s bookmark bar (Chrome, Firefox & Safari). When you access an article, simply click on the icon to retrieve article level metrics where available. A caveat mentioned on their website is that it only works on PubMed, ArXiv, or articles containing a DOI. Also data from Twitter is only available since July 2011.
Video – how to install the altmetric bookmarklet https://www.youtube.com/watch?v=_Alsg7AyrhM
- Almetric Explorer is a subscription based tool that allows more browsing, filtering and reporting options.
- Plum Analytics is another subscription based service aimed at institutions and publishers.