Assessment of factors influencing the citation level of scientific publications in the field of sport and physical activity
DOI:
https://doi.org/10.15561/physcult.2025.0104Keywords:
Web of Science, Sport Sciences, citation, analysis, sourcesAbstract
Background and Study Aim. The citation level of scientific publications is a key indicator of their scientific impact and significance. With the increasing number of publications in the field of sport and physical activity, it is crucial to identify the factors influencing citation levels. The aim of this study is to analyze the relationship between the number of cited references and the citation level of publications. Additionally, the study examines the impact of access mode (open or closed) and publication type on citation metrics.
Materials and Methods. The study is based on the analysis of bibliographic data from the Web of Science database covering the period from 2015 to 2024. The dataset includes 10,000 publications related to sport and physical activity. Metrics such as Cited Reference Count (NR), Times Cited (TC), and Total Times Cited (Z9) were used to evaluate the relationship between the number of cited references and the citation level of publications. The study also analyzed the impact of access mode (open or closed) and publication type (Article, Review, Other) on citation metrics. Data processing involved descriptive statistical methods and nonparametric tests, including Spearman's correlation and the Kruskal-Wallis test.
Results. The analysis of bibliographic data revealed the following patterns. The citation level of publications is associated with the number of cited references (NR); however, the correlation remains weak (Spearman's coefficients: 0.1329 for TC and 0.1342 for Z9). The average number of cited references for highly cited articles is 69.56, with a maximum value of 766. Open-access publications exhibit significantly higher citation levels. Among them, the "Green Accepted" (7.19%) and "Green Published, hybrid" (4.45%) formats make a substantial contribution. However, the majority of publications (60.14%) fall into the "Unknown" category, limiting the assessment of their accessibility. An analysis of publication types showed that Review and Article formats have the greatest impact. These account for 21.81% and 75.29% of highly cited works, respectively. In the high-citation category, most publications fall into the "Moderate" citation subcategory (52.5%), followed by "Low High" (46.21%), while only 1.29% are in the "High" and "Very High" subcategories. A temporal analysis demonstrated that the highest number of highly cited publications was from 2015 (1,890 publications), followed by a decline in subsequent years. The lowest citation levels were observed for 2024 publications (only 18 articles), likely due to insufficient time for citation accumulation. These results confirm the influence of factors such as accessibility, publication type, and temporal characteristics on the citation metrics of publications in the field of sport and physical activity.
Conclusions. The study highlights the importance of a comprehensive approach to analyzing factors influencing the citation levels of scientific publications. Considering contextual characteristics such as publication accessibility, type, and temporal aspects is key to understanding the mechanisms underlying citation dynamics. The findings provide a foundation for developing strategies to enhance the scientific significance of publications, including optimizing article structure, selecting appropriate access models, and accounting for disciplinary specificity. Further research in this area will deepen the understanding of the relationships between various factors and citation levels, contributing to improved quality and visibility of scientific work.
References
Concise Guide to APA Style. 7th Edition. American Psychological Association; 2019.
Elsevier. Guidelines for authors. [Internet]. Elsevier; 2024 [updated 2024 Jun; cited 2024 Sep 28]. Available from: https://static.elsevier.es/norm_orga/035normas_eng.pdf
Wiley. Author Guidelines. [Internet]. Wiley; 2024 [updated 2024 Jun; cited 2024 Sep 28]. Available from: https://onlinelibrary.wiley.com/page/journal/2688268x/homepage/author-guidelines
Springer. References. [Internet]. Springer; 2024 [updated 2024 Jun; cited 2024 Sep 28]. Available from: https://www.springer.com/gp/authors-editors/authorandreviewertutorials/howtopeerreview/references/10286412?srsltid=AfmBOop_ZdFQJBVDHY9kbJDwxDpvs5V-bj7NAHQLI6yM20y5b79TtO7H
Iermakov S, Yermakova T, Wnorowski K, Bensbaa A. Beach volleyball athlete training trends of Russian-language scientific resources: a systematic review. Physical Education of Students, 2021;25(5): 319–338. https://doi.org/10.15561/20755279.2021.0508
Iermakov S, Podrigalo L, Podrihalo O, Yermakova T, Jagiello W. Means and methods of physical activity in the context of prevention and treatment of Alzheimer’s disease (analysis of Russian-language scientific resources) and the perspective of implementing the unique achievements of the “Polish School of Safe Falling”. Archives of Budo, 2022;18:121–143.
Tykhonkova I. References are the important indicator of articles` quality. How to escape an extra work. In: Ukrainian Science in the Global Information Space, 2015. P. 100–106. https://doi.org/10.13140/RG.2.1.4176.5842
Peh WCG, Ng KH. Preparing the references. Singapore Medical Journal. 2009;50(7): 659–661.
Blicq RS, Moretto LA. Writing Reports to Get Results: Quick, Effective Results Using the Pyramid Method. 1st ed. Wiley; 2015. https://doi.org/10.1002/9781119134626
Staiman Avi. The Bibliography - Formal Requirement or Essential Ingredient?; [Internet]; 2017. [cited 2023 Nov 15]. Available from: https://www.researchgate.net/publication/323279708_The_Bibliography_-_Formal_Requirement_or_Essential_Ingredient
Jawaid SA, Jawaid M. How to write introduction and discussion. Saudi Journal of Anaesthesia, 2019;13(Suppl 1), S18.
Fox CW, Paine CT, Sauterey B. Citations increase with manuscript length, author number, and references cited in ecology journals. Ecology and Evolution, 2016;6:7717–7726.
Leydesdorff L, Wouters P, Bornmann L. Professionaland citizen bibliometrics: Complementarities and ambiva-lences in the development and use of indicators - A state-of-the-art report. Scientometrics, 2016; 109: 2129–2150.
Baumgartner SE, Leydesdorff L. Group-BasedTrajectory Modeling (GBTM) of citations in scholarly litera-ture: Dynamic qualities of “transient” and “sticky knowledgeclaims.” Journal of the Association for Information Scienceand Technology, 2014;65: 797-811. https://doi.org/10.1002/asi.23009
Aksnes DW, Langfeldt L, Wouters P. Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories. SAGE Open, 2019;9(1): 215824401982957. https://doi.org/10.1177/2158244019829575
Hui A, Rains LS, Todd A, Boaz A, Johnson S. The accuracy and accessibility of cited evidence: a study examining mental health policy documents. Social Psychiatry and Psychiatric Epidemiology, 2020;55(1): 111–121. https://doi.org/10.1007/s00127-019-01786-8
Torres Á. Quality of the references [Internet]; 2016 Oct 23 [cited 2023 Nov 15]. Available from: https://doi.org/10.3916/school-of-authors-021
Santini A. The Importance of Referencing. Journal of Critical Care Medicine (Universitatea De Medicina Si Farmacie Din Targu-Mures), 2018;4(1): 3–4. https://doi.org/10.1515/jccm-2018-0002
Kumar A. Quality of references reveal the merit of the research. Journal of Indian Society of Periodontology, 2022;26(6): 519–520. https://doi.org/10.4103/jisp.jisp_410_22
Sanchez-Ramos L, Lin L, Romero R. Beware of references when using ChatGPT as a 356source of information to write scientific articles. American Journal of Obstetrics and Gynecology, 2023;229(3):356–357. https://doi.org/10.1016/j.ajog.2023.04.004
McGowan A, Gui Y, Dobbs M, Shuster S, Cotter M, Selloni A, et al. ChatGPT and Bard exhibit spontaneous citation fabrication during psychiatry literature search. Psychiatry Research, 2023;326: 115334. https://doi.org/10.1016/j.psychres.2023.115334
Walters WH, Wilder EI. Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 2023;13(1): 14045. https://doi.org/10.1038/s41598-023-41032-5
Hueber AJ, Kleyer A. Quality of citation data using the natural language processing tool ChatGPT in rheumatology: creation of false references. RMD Open, 2023;9(2): e003248. https://doi.org/10.1136/rmdopen-2023-003248
Comeau DC, Wei CH, Islamaj Doğan R, Lu Z. PMC text mining subset in BioC: about 3 million full text articles and growing. Bioinformatics,; 2019.
Food Technology and Biotechnology. Guidelines for Writing References. [Internet]. 2023 [updated 2023 Jun; cited 2024 Sep 12]. Available from: https://hrcak.srce.hr/upute/upute_reference_Food_Technology_and_Biotechnology.pdf
Anger J. Should there be a recommended limit to the number of references in a scientific article? Sao Paulo Medical Journal, 1999;117(1): 42–43. https://doi.org/10.1590/S1516-31801999000100010
Elsevier. Article Types [Internet]. 2023 [updated 2023 Jun; cited 2024 Sep 02]. Available from: https://legacyfileshare.elsevier.com/promis_misc/jnrt-article-types.pdf
Liang Z, Mao J, Lu K, Li G. Finding citations for PubMed: a large-scale comparison between five freely available bibliographic data sources. Scientometrics, 2021;126(12): 9519–9542. https://doi.org/10.1007/s11192-021-04191-8
Rahayu NI, Suherman A, Muktiarni. A Bibliometric Analysis Of Physical Activity Research Using Vosviewer. Journal of Engineering Science and Technology, 2023;18: 206–216.
Chen L, Ren T, Tan Y, Li H. Global trends of research on depression in breast cancer: A bibliometric study based on VOSviewer. Frontiers in Psychology, 2022;13: 969679. https://doi.org/10.3389/fpsyg.2022.969679
Rahayu N indri, Bachari A dutha, Muktiarni M, Maryanti RINA. Information and communication technology (ict) intervention targeting physical activity and diet behaviors in people with disabilities: Vosviewer mapping analysis. Journal of Engineering Science and Technology, 2023;18: 164–175.
Kulczycki E, Holowiecki M, Taskin Z, Krawczyk F. Citation patterns between impact-factor and questionable journals. Scientometrics, 2021;126(10): 8541–8560. https://doi.org/10.1007/s11192-021-04121-8
Nalbant A. Does altmetric score affect the impact factor of anatomy journals? European Journal of Anatomy, 2022;26(3): 263–272. https://doi.org/10.52083/UXMV4266
Lyne J, Doherty AM, Hallahan B. Impact factor and future directions for the Irish Journal of Psychological Medicine. Irish Journal of Psychological Medicine, 2023;40(3): 321–322. https://doi.org/10.1017/ipm.2023.42
Gazendam A, Cohen D, Morgan S, Ekhtiari S, Ghert M. Quotation Errors in High-Impact-Factor Orthopaedic and Sports Medicine Journals. JBJS Open Access, 2021;6(3): e2100019. https://doi.org/10.2106/JBJS.OA.21.00019
Lugaz N. Space Weather Bibliometrics for 2022: Going Beyond Impact Factor. Space Weather-the International Journal of Research and Applications, 2022;20(9): e2022SW003276. https://doi.org/10.1029/2022SW003276
Ho YS, Giordano V, Mauffrey C, V. Giannoudis P. Trends of impact factor contributors to the Injury Journal: A bibliometric analysis. Injury-International Journal of the Care of the Injured, 2024;55(3): 111255. https://doi.org/10.1016/j.injury.2023.111255
Silva D de O, Taborda B, Pazzinatto MF, Ardern CL, Barton CJ. The Altmetric Score Has a Stronger Relationship With Article Citations Than Journal Impact Factor and Open Access Status: A Cross-sectional Analysis of 4022 Sport Sciences Articles. Journal of Orthopaedic & Sports Physical Therapy, 2021;51(11): 536–542. https://doi.org/10.2519/jospt.2021.10598
Ariza-Guerrero AM, Blazquez JS. Evolution of number of citations per article in Materials Science: possible causes and effect on the impact factor of journals. Scientometrics, 2023;128(12): 6589–6609. https://doi.org/10.1007/s11192-023-04863-7
Martin RL, Davenport TE, Fraser JJ, Sawdon-Bea J, Carcia CR, Carroll LA, et al. Ankle Stability and Movement Coordination Impairments: Lateral Ankle Ligament Sprains Revision 2021. Journal of Orthopaedic & Sports Physical Therapy, 2021;51(4): CPG1–CPG80. https://doi.org/10.2519/jospt.2021.0302
Wut TM, Xu J, Wong S mun. Crisis management research (1985?2020) in the hospitality and tourism industry: A review and research agenda. Tourism Management, 2021;85: 104307. https://doi.org/10.1016/j.tourman.2021.104307
Guest NS, VanDusseldorp TA, Nelson MT, Grgic J, Schoenfeld BJ, Jenkins NDM, et al. International society of sports nutrition position stand: caffeine and exercise performance. Journal of the International Society of Sports Nutrition, 2021;18(1): 1. https://doi.org/10.1186/s12970-020-00383-4
Pranckutė R. Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World. Publications, 2021;9(1): 12. https://doi.org/10.3390/publications9010012
Powell KR, Peterson SR. Coverage and quality: A comparison of Web of Science and Scopus databases for reporting faculty nursing publication metrics. Nursing Outlook, 2017;65(5): 572–578. https://doi.org/10.1016/j.outlook.2017.03.004
Martín-Martín A, Orduna-Malea E, Thelwall M, Delgado López-Cózar E. Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories. Journal of Informetrics, 2018;12(4): 1160–1177. https://doi.org/10.1016/j.joi.2018.09.002
Stahlschmidt S, Stephen D. Comparison of Web of Science, Scopus and Dimensions databases. KB Forschungspoolprojekt; 2020.
Web of Science [Internet]; 2023 Oct 23 [cited 2023 Nov 15]. Available from: https://access.clarivate.com/login?app=wos
Yang S, Han R. Breadth and depth of citation distribution. Information Processing & Management, 2015;51(2): 130–140. https://doi.org/10.1016/j.ipm.2014.12.003
Lin Z, Wang Y, Li H. Depth and Breadth of Research Area Coverage and Its Impact on Publication Citation: An Analysis of Bibliometric Papers. 2023; https://doi.org/10.48550/ARXIV.2311.06785
Stern DI, Tol RSJ. Depth and breadth relevance in citation metrics. Economic Inquiry, 2021;59(3): 961–977. https://doi.org/10.1111/ecin.12994
Wang SQ, Wang JX, Zhang C, Sun FH, Xie YJ, Jiang W, et al. What You Should Know About Osteoarthritis Rehabilitation: A Bibliometric Analysis of the 50 Most-Cited Articles. Geriatric Orthopaedic Surgery & Rehabilitation, 2020;11: 2151459320973196. https://doi.org/10.1177/2151459320973196
ELsinghorst ThAM. Analysis of the 96 most often cited articles published in veterinary journals in 2002 and 2003. Veterinary Quarterly, 2005;27(4): 183–189. https://doi.org/10.1080/01652176.2002.9695199
Gan Y, He Q, Li C, Alsharafi BLM, Zhou H, Long Z. A bibliometric study of the top 100 most-cited papers in neuroendocrine prostate cancer. Frontiers in Oncology, 2023;13: 1146515. https://doi.org/10.3389/fonc.2023.1146515
Zhang Z, Van Poucke S, Goyal H, Rowley DD, Zhong M, Liu N. The top 2,000 cited articles in critical care medicine: a bibliometric analysis. Journal of Thoracic Disease, 2018;10(4): 2437–2447. https://doi.org/10.21037/jtd.2018.03.178
Liu Z, Ning W, Liang J, Zhang T, Yang Q, Zhang J, et al. Top 100 cited articles in the thromboangiitis obliterans: a bibliometric analysis and visualized study. European Journal of Medical Research, 2023;28(1): 551. https://doi.org/10.1186/s40001-023-01540-6
Dougherty MR, Horne Z. Citation counts and journal impact factors do not capture some indicators of research quality in the behavioural and brain sciences. Royal Society Open Science, 2022;9(8): 220334. https://doi.org/10.1098/rsos.220334
Nieminen P, Carpenter J, Rucker G, Schumacher M. The relationship between quality of research and citation frequency. BMC Medical Research Methodology, 2006;6(1): 42. https://doi.org/10.1186/1471-2288-6-42
Dorta-González P, Dorta-González MI. Citation differences across research funding and access modalities. The Journal of Academic Librarianship, 2023;49(4): 102734. https://doi.org/10.1016/j.acalib.2023.102734
Yun JJ, Liu Z, Jeong E, Kim S, Kim K. The Difference in Open Innovation between Open Access and Closed Access, According to the Change of Collective Intelligence and Knowledge Amount. Sustainability, 2022;14(5): 2574. https://doi.org/10.3390/su14052574
Castillo M. Citations and Open Access: Questionable Benefits: Fig 1. American Journal of Neuroradiology, 2009;30(2): 215–216. https://doi.org/10.3174/ajnr.A1325
Momeni F, Fraser N, Peters I, Mayr P. From closed to open access: A case study of flipped journals. 2019; https://doi.org/10.48550/ARXIV.1903.11682
Weale AR, Bailey M, Lear PA. The level of non-citation of articles within a journal as a measure of quality: a comparison to the impact factor. BMC Medical Research Methodology, 2004;4(1): 14. https://doi.org/10.1186/1471-2288-4-14
Kousha K, Thelwall M. Factors associating with or predicting more cited or higher quality journal articles: An Annual Review of Information Science and Technology (ARIST) paper. Journal of the Association for Information Science and Technology, 2024;75(3): 215–244. https://doi.org/10.1002/asi.24810
Abramo G, D’Angelo CA. Evaluating research: from informed peer review to bibliometrics. Scientometrics, 2011;87(3): 499–514. https://doi.org/10.1007/s11192-011-0352-7
Teplitskiy M, Duede E, Menietti M, Lakhani KR. How status of research papers affects the way they are read and cited. Research Policy, 2022;51(4): 104484. https://doi.org/10.1016/j.respol.2022.104484
Thelwall M, Kousha K, Stuart E, Makita M, Abdoli M, Wilson P, et al. In which fields are citations indicators of research quality? Journal of the Association for Information Science and Technology, 2023;74(8): 941–953. https://doi.org/10.1002/asi.24767
Lafia S, Thomer A, Moss E, Bleckley D, Hemphill L. How and Why Do Researchers Reference Data? A Study of Rhetorical Features and Functions of Data References in Academic Articles. Data Science Journal, 2023;22: 10. https://doi.org/10.5334/dsj-2023-010
Nightingale JM, Marshall G. Citation analysis as a measure of article quality, journal influence and individual researcher performance. Radiography, 2012;18(2): 60–67. https://doi.org/10.1016/j.radi.2011.10.044
Dardas LA, Sallam M, Woodward A, Sweis N, Sweis N, Sawair FA. Evaluating Research Impact Based on Semantic Scholar Highly Influential Citations, Total Citations, and Altmetric Attention Scores: The Quest for Refined Measures Remains Illusive. Publications, 2023;11(1): 5. https://doi.org/10.3390/publications11010005
Downloads
Published
How to Cite
Issue
License
Copyright (c) 2025 Sergii Iermakov, Georgiy Korobeynikov
This work is licensed under a Creative Commons Attribution 4.0 International License.
Abstract views: 606 / PDF downloads: 249
