Original Research

Impact of a Fulbright award: A bibliometric analysis of persistence

Marie Hastings-Tolsma, Charlene Downing, Annie Temane, L. Amy Giles, Jean L. Hillyer, Sean C. Beatty

Received: 08 July 2024; Accepted: 04 Nov. 2024; Published: 05 Mar. 2025

Copyright: © 2025. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background: While hundreds of Fulbright awards have been given, little is known about the impact of such engagement despite a goal of increased research partnership and collaboration.

Aim: The extent and impact of a Fulbright award was explored by examining referencing of primary collaborative publications.

Setting: Seven databases and two alternative sources from 2013 to 2023 were reviewed.

Methods: Co-citation analysis identified pairs of referenced articles. Subsequently, a bibliometric approach was used to quantitatively and visually capture and analyse publications using data visualisation software.

Results: A search of sources found 773 citations citing the 16 primary works. Following the elimination of duplicates, 273 publications remained. Also examined was the non-scientific downstream noted in social media (n = 66). Based on co-citation analysis, there was a sharp uptick in the utilisation of primary citations (n = 273) compared to a 2019 analysis (n = 42). Journal Impact Factors of citing works had a high of 5.379 from 2.079 in 2019. Primary citations in open access journals demonstrated greater referencing, and the COVID-19 pandemic exacerbated utilisation of some works. Citing works focussed on three clusters: compassion fatigue, birth stories and Ubuntu. Most citing works originated from South Africa and the United States of America and consisted of multidisciplinary investigators with interfacility alliance and team science engagement. Nursing or midwifery were the main disciplines of first authors in citing articles (n = 153).

Conclusion: Co-citation analysis and downstream use of publications in social media provided evidence of the impact of a Fulbright award on scholarship with persistence over time.

Contributions: Fulbright awards promote collaborative teamwork between disciplines and is of clear benefit to scientists.

Keywords: bibliometrics; bibliometric analysis; bibliometric mapping; altmetrics; infographics; data visualisation; citation analysis; Fulbright award.

Introduction

Fulbright awards, an international educational exchange programme, were initiated in 1946 through the efforts of Senator William Fulbright of the United States of America (USA) (U.S. Department of State Bureau of Educational and Cultural Affairs (ECA) 2023). Engagement offers the opportunity to increase awareness of cultural diversity through intensive immersion in teaching, research and/or community engagement while promoting partnership and collaboration.

Designed to increase understanding between those from the USA and other countries, hundreds of Fulbrights have been awarded though relatively little is known about the impact and reach of these engagements. Over the past couple of decades, there have been numerous reports regarding the experience of the Fulbright awardee and results from approved project work. However, there is a paucity of knowledge regarding the extent and the impact of the award over time, as well as whether collaborations are sustained following the exchange period. The need for such determination has been noted (Downing et al. 2021; Van Woerkom 2010).

One of the investigators (M.H.-T.) was a Fulbright USA Faculty Scholar to the Republic of South Africa (RSA). The research-teaching award was for 1 year (2012–2013) with the University of Johannesburg Department of Nursing hosting the scholar. The purpose of this project was to ascertain the extent and impact of the Fulbright award by examining primary publications that emanated from ongoing collaboration using bibliometric analyses. A secondary purpose was to examine non-scientific downstream citations as found in social media sources. Finally, investigators sought to compare dissemination efforts from works published between 2013 and 2019 with the 2013 and 2023 analyses. In summary, this work advances more traditional bibliometric analytic work specific to a focussed area of scholarship.

Bibliometrics

Bibliometrics uses citations from publications to gauge the influence of these published works on other scientific publications (Bornmann & Haunschild 2018). The process uses statistical analyses to then provide a visual graphic (McBurney & Novak 2002). The approach has been deemed a reliable method to conduct quantitative and empirical investigation of previously published works in any field (Ellegaard & Wallin 2015), with research questions answered by extracting publication data (Belter 2015). The result is a type of infographic that gives a visual snapshot of complex data (Thompson & Walker 2015), enables scientists to conduct macroscopic and microscopic analyses of large numbers of publications (Alfonzo, Sakraida & Hastings-Tolsma 2014) and facilitates ease in understanding the impact of the data (Smiciklas 2012).

There has been an explosion of bibliometric works over the past 25 years, in part because of access to relatively easy-to-use mapping tools and support from academic librarians (Cox et al. 2019). For example, few studies have examined the use of bibliometrics in nursing or related health sciences with detailed content analysis until more recently. Kokol and Vošner (2019) reviewed the use of bibliometrics in nursing and demonstrated a dramatic recent increase with a positive trend in the number of publications, diversity of journals and number of countries (with greater production in G7 countries and those with successful economies). Internationally authored bibliometric publications were cited more than those written by authors from just one country. Findings were consistent with prior research (Kokol et al. 2014; Železnik, Blažun Vošner & Kokol 2017).

Scientists in the academic environment have been particularly interested in the use of bibliometrics as a means of documenting the impact and reach of scholarly works – beyond the more traditional metrics of Journal Impact Factor (JIF), Eigenfactor, m-quotient and h-, i10- or g-indices. Interestingly, universities (Ndwandwe et al. 2021) and grant funding agencies (Gunashekar, Wooding & Guthrie 2017; Recio-Saucedo et al. 2022) have increasingly used bibliometrics to assess scholarly output by faculty.

A corollary in bibliometrics that has received far less attention is how citations impact social media sources and lay publications – the ‘altmetrics’ (Garcia-Villar 2021). Web-based altmetric tools provide a more nuanced story about the use and impact of published works (Champieux 2015). Such data give evidence of how and where published works are shared and discussed and by whom, though meaning has not yet been fully clarified (Díaz-Faes, Bowman & Costas 2019). Altmetrics have gained increased acceptance across institutions and are increasingly used as a tool to document scholarship, supporting professional advancement efforts such as promotion and tenure. A robust evaluation of such works provides an important complement to traditional methods in determining scholarly output and impact (Baheti & Bhargava 2017) and gives evidence of heightened outreach and engagement (Wang et al. 2017). Determining varied social media metrics, such as downloads, reads, tweets, Facebook posts and news mentions, is increasingly doable as available tools (e.g. Altmetrics, Google Scholar, Plum X, ResearchGate and Mendeley) have proliferated. Caution needs to be exercised; however, as these alternative metric tools need relevance to research objectives (Bornmann & Haunschild 2018).

This research aimed to compile and analyse the scholarly articles that emanated from a Fulbright award. The research objectives were:

1. What were the number of published works citing the primary publications published between 2013 and 2023?

2. What was the growth trajectory in the number of publications citing primary works?

3. What were the journal distribution, JIFs, keywords, major subject clusters, geographic distribution of the primary author, disciplines and research methodology of studies citing primary, seminal works?

4. What were the non-scientific uses of primary citations as noted in social media?

5. What were the bibliometric mapping and non-scientific use of primary citation differences between the 2019 and 2023 analyses?

Research methods and design

Study design

A bibliometric analysis was conducted to examine the citation of articles emanating from the collaboration of researchers engaged in work as the result of a Fulbright award (N = 16) (Table 1). Citation analyses examined primary referring documents for citations and provided a means of assessing both publication output and the influence on the defined field. Specifically, co-citation analysis was used to track pairs of studies that were cited together in referent, source articles. When a pair of studies are co-cited by varied authors, clusters of research form. Co-citation analysis is prospective and dependent on the development of an academic field (Small 1973). Where citations were noted, it was believed to reflect confidence regarding the impact of identified publications (e.g. articles, books and other works) on science, as a whole (Belter 2015). Specifically, this research analysed the merit and impact where primary citations were sourced – a widely used bibliometric mapping approach (Schaer 2013). Further, PlumX and Altmetric were used to examine the non-scientific downstream use of primary publications in social media and lay literature.

Identification of co-citations

Sixteen primary studies were searched for citation usage from 01 January 2013 through 15 January 2023. PubMed MEDLINE was searched to verify all author names, article titles, journal titles and any other missing parts of the 16 published articles. Databases vary in the number of journals where articles are indexed, abstracted or published in full text, resulting in the need to search multiple databases. The fact that not all databases have a complete record of information regarding published works necessitated the need to conduct a wide search of the literature.

A total of seven databases and two alternative sources were searched to capture the articles citing the 16 published primary works. The seven databases reviewed were Scopus, Web of Science, PubMed, Embase, CINAHL (Cumulative Index to Nursing and Allied Health Literature), Papers First and Proceedings First. Alternative sources searched were Google Scholar and ResearchGate. ResearchGate produced 224 citations, far exceeding the numbers produced by academic databases.

Searching Scopus and the Web of Science Core Collection databases, author name variations for the 16 journal citations were searched, including author name misspellings that were identified. The search was conducted for all articles by each author during the search period confirming the author’s name. In addition, variations in title words reflecting British or American English word spellings were also searched. The search process below was followed:

• Last name I1
• Last name I1I2
• Last name I1I2I3
• Last name First name
• Last name First name I2I3
• Cited reference search.

PubMed was searched by focussing on author and journal name variations. The process produced citations that indicated which authors cited one of the 16 published articles and in which journal each article was published. The search process included all additional misspelled author or journal names that appeared in the Web of Science search process:

• Last name = Author family name as provided by the PI
• First name = Author given name as discovered through the database search or from the full text of the articles
• I1 = Author first initial
• I2 = Author second initial
• I3 = Author third initial

The cited reference search process was conducted using each article’s first author. If nothing was found, the search continued with the second author and then the remaining authors as appropriate.

Embase and CINAHL were searched using the same search procedure described for Scopus and Web of Science ensuring that no citations of the 16 original articles were missed. Results were found in both databases.

Google Scholar and ResearchGate provided access to article citations and in some cases full-text articles, and both produced results. In both instances, author names and article titles were searched for all 16 original articles. Notably, ResearchGate produced the most results from all databases and alternative sources. This may be attributable to the ability of authors to self-index published articles. Presently, the breadth of the results provided a wider international scope than publisher-curated databases:

1. The search strategy for the selection of studies related to the use of the 16 primary works can be seen in Figure 1. All searching and identification of citing works was conducted by the librarian co-author.

FIGURE 1: Search strategy and selection of articles related to the use of 16 primary articles.

Identification of non-scientific downstream citations

A noted gap in bibliometric literature is the examination of the downstream use of publications as noted through social media (Kokol & Blažun Vošner 2019). Such use can be determined using altmetrics or ‘alternative metrics’, which assist researchers in understanding how published works are being used and communicated to other end-users (Dardas et al. 2019).

To examine the use of each of the 16 primary published works in lay publications, grey literature, research blogs, mentions, reads and downloads, two altmetric services were utilised for analyses. These included PlumX Analytics and Altmetric, which were selected because the authors had access to academic libraries in possession of a licence for use. Such licence provided the altmetrics for the identified scholarly and creative works.

PlumX Analytics provided comprehensive item-level metrics, which provided insight into how individuals interacted with a given publication in an online environment. PlumX best collects Mendeley readership (i.e. users who have added an article to their library, demographic insights such as geographic information, discipline and academic status) (Ortega 2018). PlumX details five categories including citations, usage, captures, mentions and social media (Plum Analytics 2023).

Research has demonstrated that Altmetric.com provides the best coverage of blog posts, news and tweets (Ortega 2018). Altmetric required a DOI (digital object identifier) to generate an identified attention score – a whole number derived from an automated, weighted algorithm. This Altmetrics Attention Score (AAS) is an aggregated indicator that serves as a proxy for the volume and nature of attention – the downstream impact and in some cases public engagement, that a work receives online. The AAS begins to be generated immediately following publication and, in effect, serves as a real-time indicator of interest in the work (Lee, Choi & Michos 2021). Altmetric created a scoring system as a complement to traditional bibliometrics and tracked the online media presence of an article by measuring and compiling the mentions an article receives across varied outlets including Facebook, X (formerly Twitter), blogs, policy sources, Wikipedia and other platforms (Altmetric 2023). Mentions of a given article are weighted and a final Altmetric score reflects a summation of these weighted mentions.

Bibliometric Mapping

Bibliometric analysis here relied on Python and R (R Core Team 2020), two programming languages fundamental to modern academic data science, bibliometrics and statistical analysis. Their open-source communities have facilitated the development of numerous tools for data aggregation, mapping and visualisation. Data manipulation was applied using the ‘tidyverse’ package (Wickham, Vaughan & Girlich 2023), a collection of R packages designed for data science. For data visualisation, the authors employed ‘ggplot2’ (Wickham 2016). The authors incorporated network diagrams using the ‘igraph’ package (Csárdi et al. 2023). This package offers robust tools for network analysis and visualisation. Lastly, the authors utilised the Maps package (v3.3.0, ‘maps’ (Becker et al. 2018)) for generating geographical maps. This package provided a comprehensive set of map data useful in giving a geographic context to the academic influence and reach of the publications analysed.

Ethical considerations

This research was reviewed on 16 March 2023, by the Baylor University Institutional Review Board and deemed exempt (IRBNet #2034964).

Results

Published works citing primary publications and growth trajectory

The 16 primary referenced works (see Table 1) were cited in 273 studies (see Figure 2). In the 2019 analysis of citations, primary studies (N = 11) had been cited 42 times (Downing et al. 2021). The citation receiving the greatest number of citations (n = 159) in the current analysis was the work related to compassion fatigue (Table 1, Paper 6) (Nolte et al. 2017). Five primary works received two or fewer citations (De Klerk, Temane & Downing 2023; Hastings-Tolsma et al. 2012, 2021; Hastings-Tolsma & Nolte 2013; Nolte, Hastings-Tolsma & Harper 2013). There was one primary work (Paper 16) that had no citations; 11 other citing publications (1, 3, 4, 7, 8, 9, 10, 11, 12, 13, 15) had eight or fewer secondary citations. Twelve (12) of the 16 primary publications had self-citations; four of the self-citations occurred within the first year following publication of the primary work.

FIGURE 2: Citation counts (n = 273) resulting from primary works (N = 16), 2013–2023.

Impact of primary citations

To determine impact, cited primary works were examined for JIF, author-identified keywords, research methods employed by citing authors, focus of major clusters, disciplinary affiliation, geographic origin of first authors and presentations (see Table 2). In addition, non-scientific downstream of primary citations was examined to determine the influence on society (see Table 3).

Journal impact factor

The initial 2019 list of primary articles (N = 11) was referenced in journals with JIFs ranging from 0.0250 to 2.079. The current comprehensive assessment of primary articles (N = 16) demonstrated JIFs ranging from unreported to JIFs of 0.08 to 5.379. Specifically, an examination of the JIFs of citations emanating from the primary referenced works (N = 16) found that the lowest impact factor was 0.08 in Douleurs – a peer-reviewed journal that serves a multidisciplinary audience of professionals involved in the management of pain. The highest JIF was 5.379 in Social Science & Medicine, which provides an international and interdisciplinary forum for the dissemination of social science research on health.

Journals with a JIF ranging from four to five were distributed in the following range: lowest: 4.072 (Review of Public Personnel Administration) to 4.865 (International Journal of Mental Health Nursing). The JIF of 4.423 occurred on seven occasions and belonged to the Journal of Clinical Nursing. Twenty-seven articles were published in a journal with a JIF ranging from 3.928 to 3.03. In this range, three articles were published in Women and Birth (JIF 3.172), two in the International Journal of Africa Nursing Sciences (JIF 3.172), two articles in Nursing Ethics (JIF 3.344) and three articles in the Journal of Nursing Management (JIF 3.325).

Keywords

In a previous study (Downing et al. 2021), the top author keywords were health (15), nurse (11), fatigue (10) and compassion (10), and these were considered the basic elements of the research niche or domain. In the current study, the general keywords demonstrated a broad overview of health and nursing and were identified as important research themes for this bibliometric analysis. High-frequency keywords were compassion fatigue (66), burnout (40), caring (11), Ubuntu (10), mental health and well-being (9) and mindfulness (6).

For the top cited studies, the keywords emanating from Paper 7 included compassion fatigue, burnout, vicarious trauma, traumatic stress, compassion satisfaction, compassionate care, coping and resilience. Twelve of the 160 studies identified the coronavirus disease 2019 (COVID-19) as a keyword. Paper 6 had 22 citations and common keywords included labour, childbirth, humanisation, caring and/or uncaring encounters, birth companion, obstetric violence and neglect. Papers 3 and 16 focussed on the theme of Ubuntu and common keywords were caring and/or caregiving, Ubuntu, nursing and/or healthcare professionals, community, quality of life and individualism.

Focus major clusters

Studies citing one of the primary referenced studies (N = 16) demonstrated one of three major clusters. The first major cluster focussed on the article ‘Compassion Fatigue in Nurses: A Meta-synthesis’ (Paper 7 published in 2017) and resulted in 160 studies. Citings increased from 24 to 160 from 2019 to 2023, respectively. The primary disciplines referencing the article were nursing, business management and allied health professionals. The second major cluster focussed on the article ‘Birth Stories from South Africa: Voices Unheard’ (Paper 6), which was published in 2018. The study was cited two times in 2019 and 22 times in 2023. The third major cluster focussed on Ubuntu and included two articles: ‘An Integrative Review of Albertina Sisulu and Ubuntu – Relevance to Caring and Nursing’ (Paper 3 published in 2016) and ‘Ubuntu the Essence of Caring and Being – a Concept Analysis’ (Paper 14 published in 2019) with 41 total citing studies. Paper 16 (published in 2022) had no citations. All but two of the primary publications included the Fulbright recipient. Figure 2 demonstrates how studies are clustered.

Study methods and design

The methodologic approach of the primary studies (n = 11) reviewed in the 2019 analyses (Downing et al. 2021) included three qualitative studies, two quantitative and six other methodologic studies (i.e. concept analyses, integrative review, etc.). Determination of the methodology used in primary works conducted after 2019 (n = 5) found the addition of one qualitative and quantitative approach each, as well as three other approaches (i.e. concept analyses, discourse).

The number of citing articles substantially increased from 42 (2013–2019) to 273 (2013–2023) with most citing studies having utilised a quantitative or qualitative methodology and most published in the Journal of Clinical Nursing and Pain Management journals. The increase in publications evinced the need to further explore the output and the impact of a Fulbright on nursing and other professions. Citations emanating from the primary referenced citations were primarily quantitative methodology (38%), followed by qualitative methodology (28%). Approximately, 38% were ‘other’ types (e.g. reviews, critical evaluations, concept analyses, discussions and opinions).

Disciplinary affiliation

Most primary studies (n = 1) had multinational (n = 13, 81%) authorship; some had multidisciplinary authorship (n = 4, 25%). Citing studies reflected diverse disciplinary affiliations. The highest number of citing studies for referenced works were authored by scholars whose disciplines included nursing (n = 123), midwifery (n = 30), psychological or social and health sciences (n = 25), medicine and other healthcare (n = 16) and education (n = 11). Studies citing primary works also included multinational (n = 57, 22%) and multidisciplinary (n = 83, 32%) authorship.

Nursing scholars cited primary works 123 times in 2023 compared to 27 times in 2019. Midwifery citing increased from 11 to 30. Other categories not represented in the 2019 analytic work included engineering, information management systems and technology, political science, policy, rural health and economics.

Geographic distribution

Referenced works in 2019 (Downing et al. 2021) were authored by scholars from the RSA, USA and the United Kingdom (UK), including Canada. All continents cited primary publications (n = 11) except South America. Ten of the publications had multinational authorship; three had multidisciplinary authorship. In our current study, the primary works (n = 16) added the engagement of a scholar from Eswatini. All but three of the original works had authorship from two or more countries.

The highest number of citing studies when considering all authors on secondary works were generated from the RSA (other than the rest of Africa) (n = 75), Africa (n = 7), USA (n = 65), Europe (n = 39), Asia (n = 36), Canada (n = 34), UK (England, Ireland only) (n = 32), Australia or New Zealand (n = 26), Middle East (n = 34), Central and South America (n = 6) and other countries (e.g. Caribbean) (n = 2). Figure 3 and Figure 4 demonstrate the geographic distribution for articles citing one of the 16 primary works published between 2013 and 2023. All countries increased their number of citations when compared to the 2019 analyses (Downing et al. 2021) with the UK and Canada increasing by 5.7- and 4-fold, respectively.

FIGURE 3: Global citation (n = 273) distribution citing 16 primary works (2013–2023).

FIGURE 4: Citation (n = 273) distribution citing 16 primary works, detailed Europe and Middle East distribution.

Presentations

Only one of the original primary publications was a presentation (see Table 1, Paper 11). Secondary citations of the original studies found three presentations. However, only one presentation (see Table 2, Hlojeng & Makura 2020) had a published abstract that appeared in the Proceedings First database.

Non-scientific downstream

Of the 16 primary publications, eight (50%) had no social media presence. There were alternative metrics for nine (56%) of the 16 primary sources. Specifically, usage metrics were available for nine of the publications, with abstract views for eight (50%) of the 16 primary sources (range 160–2592). There were link-outs for eight works, which ranged from 42 to 1717. Full-text views were noted for seven publications (range 0–1724), ‘export save’ was noted for eight (range 25–333), and ‘readers’ was noted for eight (range 14–143). There was one primary work picked up by five news outlets, and nine mentions of a work. Overall, nine primary citations had usage and citations; eight had captures. There were no reports of citations being referenced in videos (e.g. YouTube) or on research blogs. Only four of the primary works (Downing & Hastings-Tolsma 2016; Hastings-Tolsma & Nolte 2014; Hastings-Tolsma, Nolte & Temane 2018; Nolte et al. 2017) had been tweeted (range 3–32). Seven of the 16 works had been posted on Facebook but only one had any likes, shares, or comments. Of the 16 primary studies, eight (50%) had an AAS score (range 1–20, mean 4.87). Notably, all but two of the 16 studies had a DOI, which is required for the generation of an AAS.

When comparing our first analysis (Downing et al. 2021) with the current work, there was a social media presence for seven of 11 primary sources with three of those publications having tweets (range 23–32). Prior bibliometric work found Facebook likes, shares or comments unchanged (i.e. only one). The current work found usage, captures and citations of the 16 works at 56% (n = 9); these alternative metrics were found in 81% (n = 9) of the works analysed in 2019 (Downing et al. 2021). Usage metrics were available for nine sources and abstract views ranged from 65 to 2592. ‘Link-outs’ ranged from 18 to 1717 with ‘export save’ for seven publications (range 24–333) and ‘reader’ captures ranging from two to 143; there were no mentions in the original analyses. There were full-text views for six of the 11 primary publications (0–1724). Table 3 details the non-scientific downstream alternative metrics for the current study of 16 publications.

Discussion

Published works citing primary publications and growth trajectory

The 16 primary referenced works were cited in 273 publications. In contrast, 42 citations emanated from 11 primary citations when the analysis was conducted in 2019 (Downing et al. 2021). The 550% increase in citations gave evidence of solid growth with the bulk of growth at or after 2017 but before 2020. The large increase in citations was particularly influenced by the referencing of three primary works (Hastings-Tolsma et al. 2018; Nolte & Downing 2019; Nolte et al. 2017) (see Table 1).

Citing of published studies tends to increase over time, reflecting a concept referred to as persistence (Nicoll et al. 2018). Long-term persistence was demonstrated in this research as noted in Figure 5. Cant and Cooper (2019) have noted that there is a ‘life cycle’ for publications with a steady accumulation of citations over the first 7 years, a constant high rate between years 4 and 7 and a reducing trend thereafter. Papers 2, 3, 7, 8, 9, 10 and 11 are 7 years or older. Seven of the 16 articles were available in open access (OA) journals. The highest citation of an article was 160 for the ‘Compassion Fatigue in Nurses’ A Meta-synthesis’ (Paper 6), which was available in an OA journal. The high citation count for that article demonstrated the development of science that requires a flourishing context (Achury-Saldana et al. 2022).

FIGURE 5: Persistence of reference studies (N = 16).

As noted, there was one primary work (Paper 16) that had no citing studies, which was likely the result of a recent publication in 2022. However, 11 other primary papers (1, 3, 4, 7, 8, 9, 10, 11, 12, 13, 15) had citation counts of eight or fewer. Seven of the 16 primary works were in OA journals (Papers 2, 6, 8, 10, 12, 13, 16) but only two of these primary works had high secondary citation counts (Papers 2 and 6). Publishing in OA journals has been demonstrated to increase citation counts though often in subsets (Langham-Putrow, Bakker & Riegelman 2021), as demonstrated in this research. Low secondary citations were undoubtedly influenced by those primary studies that had a narrow focus, lack of availability in OA journals and/or publication in low or unreported JIF journals. Such restrictions clearly impacted larger access and influence.

Finally, 12 of the 16 primary publications had self-citations; only four of the self-citations occurred within the first year following the publication of the primary work. There have been negative connotations with self-citing, but it has also been noted to be characteristic of highly productive authors (Mishra et al. 2018) with a self-citation rate of approximately 5% in the fields of science and technology (Kacem, Flatt & Mayr 2020). While our self-citation rate was 12% and higher than was anticipated, most of the citations resulted from student mentorship and were reflective of productive, sustained and leading-edge efforts (Cooke & Donaldson 2014).

Impact of primary citations

To determine impact, cited primary works were examined for JIF, author-identified keywords, research methods, focus of major clusters, disciplinary affiliation, geographic origin of first authors and presentations (see Table 2). In addition, non-scientific downstream of primary citations was examined to determine influence on society (see Table 3).

Journal impact factor

The primary articles were referenced in journals with the highest JIF of 2.079 in the 2019 analysis (Downing et al. 2021). The highest JIF in the current study was 5.379 demonstrating a clear increase in being published in journals with a higher JIF and where there were a greater number of articles in the journal. Although JIF has long been regarded as the best and most impartial instrument for evaluating the prestige of a journal, there are still debates over how to calculate and interpret the score (Cooper 2015). There is agreement, however, that where there is an increase in the JIF, it indicates the journal is improving with greater numbers of researchers citing studies from the journal (Sharma et al. 2014). Journal Impact Factor, calculated annually, has many methodological flaws though academics are frequently evaluated against JIF for promotion, tenure and renumeration (Siler & Larivière 2022).

Keywords

Keyword frequency in citing references provided a good indication of the subject fields under study (Ellegaard & Wallin 2015). When comparing keywords between the initial study (Downing et al. 2021) and the current study, findings demonstrated that health, nursing and factors impacting nurses (e.g. compassion, fatigue, burnout, caring, Ubuntu and COVID-19) were significant objects of both bibliometric analyses. The keywords emanating from Paper 7 included compassion fatigue, burnout, vicarious trauma, traumatic stress, compassion satisfaction, compassionate care, coping and resilience. Twelve of the 160 studies identified COVID-19 as a keyword. Paper 6 had 22 citations and common keywords included labour, childbirth, humanisation, caring and uncaring encounters, birth companion, obstetric violence and neglect. Papers 3 and 16 focussed on the theme of Ubuntu and common keywords were caring and uncaring, Ubuntu, nursing and/or health professionals, community, quality of life and individualism. The frequency of these keywords reflected bursts of attention and underscored the strength of keywords and frequency of citation (Huang et al. 2020). As noted, the number of citations appearing in large numbers of studies gives proof of the influence of the original Fulbright award. Such notation documents the impact of the award in the expansion of foundational knowledge in a targeted area.

Focus of major clusters

The three clusters in this review of studies were ‘Compassion Fatigue in Nurses’: A Meta-synthesis’, ‘Birth Stories from South Africa: Voices Unheard’ and ‘Ubuntu’ (‘An Integrative Review of Albertina Sisulu and Ubuntu – Relevance to Caring and Nursing’ and ‘Ubuntu the Essence of Caring and Being – a Concept Analysis’). In toto, these three clusters reflected strong Fulbright and university alliance and were cited 273 times. The publications with the highest citation counts (Paper 6, 160 secondary citations; Paper 5, 22 secondary citations; Paper 2, 21 secondary citations; Paper 14, 21 secondary citations) had the greatest impact for a total of 224 secondary citations. These major clusters represent scientific topics and were easily recognised in the clusters of publications surrounding these works (Šubelj, Van Eck & Waltman 2016).

Research methodology

It was evident in the 16 primary works cited in 273 studies that Paper 6 (‘Compassion Fatigue in Nurses: A Meta-Synthesis’ by Nolte et al. 2017) had more citations. There was a plethora of methodologies that cited Paper 6 (i.e. qualitative, quantitative, discourse, systematic review, meta-analyses, cross-sectional and others); qualitative and quantitative methodologies were most common. There was demonstrated uptake in varied healthcare subspecialties with a significant impact on methodological design. Different research methods can use a variety of techniques and can also be used to find possible research partners for future studies. Ellegaard and Wallen (2015) argue that the use of different methods assists researchers to be aware of new trends in bibliometric methods.

Disciplinary affiliation

The comparison of disciplines with the number of citing studies for referenced works increased significantly. The diversification of disciplines in the current study was largely related to the expansion of disciplines researching COVID-19. Original research on coronavirus focussed on virology, medicine and infectious disease prior to the COVID-19 pandemic. After the pandemic, new research disciplines included sociology, psychology, environmental engineering, management and public administration (Abideen 2022). This mirrors the trend of increasing research on the social and psychological impacts of the pandemic as seen in the increase in compassion fatigue. Disciplines citing the Nolte et al. (2017) article on compassion fatigue increased from six in 2019 to 23 in 2023.

Nursing and midwifery remained the disciplines with the highest citing number of primary works (n = 180) and a significant increase from 27 citations in 2019. However, interdisciplinary collaborations increased to include industrial psychology and management, physiotherapy, occupational health and psychiatry. Of all citing studies, 32% (n = 83) were multidisciplinary. Publications demonstrated a pronounced expansion in the range of disciplines engaged in collaborative scientific efforts. Increased multidisciplinarity has been found to have a positive effect on impact and knowledge creation though there has been a suggestion that scientists may be more reluctant to site heterodox studies viewed as challenging or too groundbreaking (Yegros-Yegros, Rafols & D’Este 2015). Finally, 22% (n = 57) of citing studies had multinational authors. These transnational publications demonstrated a greater number of citations, which has also been noted in other bibliometric research (Adams & Gurney 2018). Multinational and multidisciplinary author affiliations are far-reaching and encourage cross-fertilisation of ideas, increasing impact and promoting development (Adams & Gurney 2018).

Geographic distribution

As in the original study (Downing et al. 2021), the RSA and USA were the two countries with the largest number of cited works. The RSA had the most citations in both analyses (n = 10, 2019; n = 64, 2023) with the USA again demonstrating the second largest number (n = 9, 2019; n = 44, 2023). This finding was likely because of the RSA and USA being the Fulbright host country and awardee country of origin, respectively.

Regardless, there was wide geographic distribution overall with an increase in the number of citations across all continents. Interestingly, distribution may have been influenced by the COVID-19 pandemic that began in the first quarter of 2020. The primary study by Nolte et al. (2017) discussed compassion fatigue and demonstrated the greatest persistence in the use of all primary works. The number of citations in the original analysis was 17 through 2019 (Downing et al. 2021); another 119 were added between 2019 and 2023 with approximately 20 articles directly addressing compassion fatigue and COVID-19. During the pandemic, the highest distribution of all research regarding compassion fatigue occurred in the USA, Canada, England and Australia (Yi et al. 2022).

The geographic growth in 2023 was likely influenced by seven OA studies, along with a number of tweets (n = 66), ResearchGate authors (n = 223) and authors’ primary country of affiliation of the RSA and USA. These factors likely had a positive influence on the number of citations (Chen et al. 2021; Gelzer et al. 2022; Tang et al. 2023). Finally, all primary articles were in English. This fact did not appear to negatively impact geographic growth apart from Central and South America where there were no citations of primary works in 2019 and three in the current analysis. These low citation counts were certainly influenced by language barriers, as well as the known exodus of scientists to the Global North (Basilio 2023).

Presentations

Only four of 19 presentations had a published abstract with a DOI, so it was not expected that there would be many references to these works. Presentations are intended for early dissemination and adoption of findings, among other goals such as networking and brainstorming (Oester et al. 2017). Researchers and conference organisers need to give greater attention to the widespread dissemination of conference abstracts, including to social media outlets with the assignment of a DOI number. The value of promissory abstracts is minimal, and it is suggested they be declined for conference presentation and dissemination (Novak & Puljak 2021). Finally, given the global explosion in conferences with an increase in ‘predatory conferences’ and the fact that there is no accreditation process for conferences, a bibliometric measure is needed to enable scholars to evaluate conference quality prior to submission of work (Makvandi, Nodehi & Tay 2021).

Non-scientific downstream

How a published work is promoted by publishers, authors and organisations through social media is important in advancing use (Gunaratne, Haghbayan & Coomes 2020). In this research, PlumX Analytics and Altmetrics were used to conduct alternative metrics for primary citations. While a wide variety of tools could have been used, the authors used these altmetric services for reasons of access and convenience. It should also be noted that the extent of use of social media (pick-up) does not reflect either the quality or impact of the work. However, the downstream use of publications does provide a complement to other conventional methods (e.g. JIF, h-, i10 or m-indices). Alternate metrics that track key usage metrics (e.g. views, mentions, shares and downloads) are known to be superior to more traditional filters in assessing scholarly impact (Patthi et al. 2017).

Of the 16 primary publications, eight (50%) had no social media presence, and it is concerning that there was so little social media or non-science visibility. Such promotion underscores lost opportunities for highlighting scholarly efforts and the potential for use by other scientists and interested parties. Where there is the use of social media in the promotion of research, improved citation is noted (Bardus et al. 2020; Klar et al. 2020).

There were alternate metrics of usage, captures and citations for nine (56%) of the 16 primary sources. Only four of the primary works (Downing & Hastings-Tolsma 2016; Hastings-Tolsma & Nolte 2014; Hastings-Tolsma et al. 2018; Nolte et al. 2017) had been tweeted (range 3–32). Seven of the 16 works had been posted on Facebook but only one had any likes, shares, and/or comments. Usage metrics were available for nine of the publications, with abstract views for eight (50%) of the 16 primary sources (range 160–2592). There were link-outs for eight works, which ranged from 42 to 1717. Full-text views were noted for seven publications (range 0–1724), ‘export save’ for eight (range 25–333) and ‘readers’ for eight (range 14–143). No mentions were found for any of the works in blog posts, newspapers, etc. Overall, nine primary citations had usage and citations; eight had captures.

Finally, only five of the 16 primary works had an AAS but other alternative metrics demonstrated that publications after 2016 and those that appeared in OA journals were most likely to be promoted through social media outlets. It is possible that other global events (e.g. pandemic and immigration patterns) were influential in pick-up and use by non-scientific communities. Results for downstream citations in non-scientific sources for the 16 primary sources are provided in Table 3.

Of note, the current study did not determine the extent to which individual authors who cited primary publications may have shared their work on social media, which is a limitation of this research. Where there are efforts to share information on social media, there is known to be a three-fold increase of shares by non-authors with greater than 50% downstream citations (Gunaratne et al. 2020). Similarly, where authors posted engagement on ResearchGate, 224 citations were produced, which far exceeded the number discerned from academic databases. Such citations demonstrate the value of social media.

In summary, it is important that both publishers and authors pay close attention to the value of disseminating research findings through social media outlets. Such outlets include social networking sites like X, Threads, YouTube video, research blogs, Google Scholar, ResearchGate, Academic.edu and LinkedIn, as well as multinational authorship with publication in OA journals (Gelzer et al. 2022; Tang et al. 2023). These sites have high numbers of active users that can be influential in the advancement of research findings. In fact, close to 70% of Americans get their news from social media because of ease of use (Shearer & Matsa 2018), and close to half of scientists follow social media to increase awareness of new discoveries and to further scientific discussions (Rainie, Funk & Anderson 2015). The value of using social media in the dissemination of publications is obvious. Such use can help scientists to have a stronger influence in a specific field (Gelzer et al. 2022) and transform how the authors communicate science to both colleagues and the broader audience (Iwasaki 2022).

Limitations

There were several limitations in this research. Firstly, self-citations were included in this research. Including self-citations has become an increasingly worrisome practice as it may unduly distort bibliometric thresholds (Baccini, De Nicolao & Petrovich 2019). Secondly, citation counts do not assess quality; rather, they are essentially a measure of popularity and further analysis is indicated. Thirdly, more recent citations may have been missed as they were not yet cited in selected databases and works not in English may have provided substantive knowledge that was not assessed. Additionally, only works with a DOI were assessed in looking at non-scientific downstream influence; works without a DOI (e.g. books, magazine articles) may have had of significant impact without determination. Future research should consider these issues. Fourthly, the use of the first author as an indicator of the geographic origin of cited works fails to fully appreciate the reach of multi-authored teams from diverse institutions. The geographic distribution of works with multiple authors should be carefully considered in future work. Fifthly, the work conducted here would benefit from a robust examination of how primary sources were used in non-scientific outlets and how such usage impacted utilisation and influence. Sixthly, the altmetric services used to determine non-scientific downstream use of the primary publications were limited to PlumX Analytics and Altmetrics.com; other platforms should be explored such as ResearchGate, LinkedIn and Google Search.

Future research

Consideration should be given to a robust examination of how the Fulbright award impacted other vital areas of engagement. This should include an examination of teaching and service involvement for both the Fulbright recipient and the host institution. It would also be useful to assess the impact of the award on generating future Fulbright applications that would further international partnerships. Replication of this research in the coming decade would be useful in providing further evidence of long-term impact. Our bibliometric work focussed on co-citation counts – or the number of times a primary source was cited. A useful strategy to consider going forward would be qualitative analysis examining the impact of publications (Cooper 2015). Qualitative assessment has been minimally used in bibliometric works but would be important to consider (Quaderi 2023). Finally, there continues to be a paucity of non-scientific downstream use of primary works – a recurring issue in the promotion of research in many fields. Evaluation of citation use in non-science journals and social media has not received significant attention and is an understudied but crucial area to consider.

Implications

Findings from this research provide a compilation of the body of publications related to one research-teaching Fulbright award. The considerable growth in the use of primary publications – particularly over the past few years – underscores the impact and reach of a Fulbright award over the ensuing decade. Results provide evidence of the significance of such an award where scientists were committed to ongoing collaboration.

The primary implication of this study is that the outcome of a Fulbright award needs to demonstrate more than geographic relocation and connection of the awardee with other community members at the host site. Considerable resources are dedicated in support of a Fulbright scholar by the host institution, the awardee’s institution and the Fulbright Foundation. There needs to be clear expectations communicated to the awardee that the financial support provided has made a clear, sustained and demonstrable impact. Fulbright awardees have a responsibility to demonstrate the larger impact of the award. Bibliometrics is but one approach to documenting the impact and reach of cross-cultural engagement. Other approaches that could be considered include public recognition of work, evidence of change in practice and procurement of extramural grant funding.

Conclusion

This study utilised co-citation bibliometric analyses to document the persistence and reach of 16 primary works that were published following a Fulbright award in 2012. The award created the opportunity for scientists from two universities in the USA and RSA to collaborate in scholarly efforts. From these collaborations between 2013 and 2023, 273 studies cited the usage of published works. While findings give solid evidence of a sustained connection between scientists, the research made no determination regarding the quality of the works. Assessment of both the quality of citing works, as well as persistence in use over the next decade, is needed to determine ongoing impact and reach. Additionally, while non-scientific downstream citations were sparse, they provided support regarding the impact and reach of the published works.

Persistence in maintaining collaborative relationships as an outgrowth of a Fulbright award is crucial to career success (Bu et al. 2018). While the quality of works emanating from this research was not assessed, persistent collaboration, multi-disciplinarity and team engagement gave evidence of the value of the Fulbright. Findings from this research paint a nuanced picture of how collaboration was of benefit to the two universities and can provide valuable insights to researchers, universities, policymakers and the Fulbright funding agency itself. Persistent scientific collaboration as an outgrowth of a Fulbright award is showcased in this project.

Acknowledgements

The authors acknowledge support from the Fulbright Foundation for awarding a Fulbright USA. Scholar Award to M.H.-T. for the period from 2012 to 2013 in South Africa, which facilitated collaboration among investigators. The editorial assistance of Susan K. Purcell, MA, is also acknowledged. Portions of this work were presented at the 6th International Conference on Nursing Science & Practice in San Francisco, CA from 12 to 15 April 2023.

The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article. The author, M.H.-T., serves as an editorial board member of this journal. The peer review process for this submission was handled independently, and the author had no involvement in the editorial decision-making process for this manuscript. The authors has no other competing interests to declare.

M.H.-T. contributed to the conceptualisation, data curation, formal analysis, methodology, data interpretation, administration and writing of the original draft, as well as review and editing. C.D. contributed to the formal analysis, methodology, data interpretation and writing of the original draft, along with review and editing. A.T. contributed to the formal analysis, methodology, data interpretation and writing of the original draft, as well as review and editing. L.A.G. contributed to the data interpretation and draft review and editing. J.L.H. contributed to the data curation, software and writing of the original draft, along with review and editing. S.C.B. contributed to the formal analysis, software and writing of the original draft, as well as review and editing.

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

The data that support the findings of this study are available from the corresponding author, M.H.T., upon reasonable request.

The views and opinions expressed in this article are those of the authors and are the product of professional research. It does not necessarily reflect the official policy or position of any affiliated institution, funder, agency or that of the publisher. The authors are responsible for this article’s results, findings and content.

References