Measuring Research Impact

Things to Consider & Limitations
There are very clear limitations to using the different types of metrics currently produced and used. Some of the limitations to the traditional metrics are as follows:

• Citation errors can lead to missing or inconsistent citations
  
• Metrics can be biased depending on the length of the research career, field of research, and format of the scholarly output
• Do not consider all forms of research output (datasets, programming language) or qualitative value of research
• Citation counts do not measure readership
• Citation counts do not reflect accuracy or authoritativeness of the research product, journal, publisher or author
• Citation counts do not include impact made outside of the scholarly community (i.e., public policy, schools environment, etc)
• Difficult to compare scores across disciplines and areas of research

Sources differ by disciplinary norms. 

Finding Journal Impact Factor: The impact factor of a journal is calculated by dividing the number of current year citations to the source items published in that journal during the previous two years.

JCR tutorial https://www.youtube.com/watch?v=VJc3PC697oc

Journal Citation Reports: Learn the Basics  http://clarivate.libguides.com/jcr

Other metrics in Journal Citation Reports:

• 5-year impact factor: The average number of times articles from the journal published in the past five years have been cited in the JCR year. It is calculated by dividing the number of citations in the JCR year by the total number of articles published in the five previous years.
• Eigenfactor Score: A reflection of the density of the network of citations around the journal using 5 years of cited content as cited by the Current Year. It considers both the number of citations and the source of those citations, so that highly cited sources will influence the network more than less cited sources. The Eigenfactor calculation does not include journal self-citations.
• Cited half-life: The median age of the items that were cited in the JCR year. Half of a journal's cited items were published more recently than the cited half-life.
• JCI-field-normalized impact: The average Category Normalized Citation Impact (CNCI) of citable items (articles & reviews) published by a journal over a recent three year period. The average JCI in a category is 1. Journals with a JCI of 1.5 have 50% more citation impact than the average in that category.

H-index (also called the Hirsch Index) was developed by a physicist to measure impact & productivity of a researcher. The formula uses number of papers and number of citing references per paper to calculate a score: It is the sum of the Times Cited divided by the number of results found. It is the most commonly used author-level metric, but others exist to account for different cases (e.g. co-authorship). It is designed to measure the overall strength of publication citations over a career and is most commonly used in STEM disciplines. Calculations rely on available publications and citations. 

There are many other variations, including:

• M-index: divides H-index by years since first published article to account for career length
• G-index: adds weight to higher-cited papers by representing g articles with at least g2 citations
• Individual H-index: gives less weight to co-authored papers by dividing h2 by the number of authors represented by the paper set
• i10-index: the number of publications with at least 10 citations; specific to Google Scholar
• Altmetrics: statistics sourced from the social Web that can be used to help you understand the many ways that your work has had an impact with other scholars, the public, policy makers, practitioners, and more. They are useful supplementary measures of impact, best used in tandem with traditional measures like citation counts. Together, the two types of metrics can illustrate the full impact of your work

Books and Book Chapters: Citations are available via Google Scholar. Qualitative-based evaluations

• Prestige of publisher (specific to discipline)
• Book reviews 

Creative Works: Impact is often qualitative, such as visual or performing arts. Factors to consider include:

• View counts for online works
• Referral URLs – what drives online traffic
• The prestige of a venue, such as for an art exhibition or performance
• Reviews

Publisher Websites

Compute your h-index

Use Web of Science to find your h-index.  Example from the Citation Report:

Use Google Scholar to find your h-index. Example from Google Scholar profile:

Stories can be equally impactful to complement contextualized metrics.

Google searches can help to identify evidence not otherwise covered by metrics

Altmetrics can help match metrics with potential audiences

• Practitioners
  • Views, downloads, syllabi use (educators)
• General public
  • Views, social media
• Industry/business
  • Patent citations, tech transfer metrics (ott.gmu.edu)
• Policymakers
  • Policy citations