Document Type : Research Paper

Authors

1 Master of Knowledge and Information Science, Allameh Tabataba’i University, Tehran, Iran.

2 Associate Professor, Department of Knowledge and Information Science, Allameh Tabataba’i University, Tehran, Iran.

3 Assistant Professor, Department of Management, Faculty of Medicine, AJA University of Medical Sciences, Tehran, Iran.

Abstract

Introduction
This study evaluates and ranks the scientific journals of psychology using the Data Envelopment Analysis (DEA) model. Data envelopment analysis (DEA) is a linear programming-based technique for measuring the relative performance of organizational units where the presence of multiple inputs and outputs makes comparison difficult. 
 DEA was originally developed to measure the efficiency of a Decision Making Units (DMU) as a whole unit, without considering its internal structure. In other words, the system is treated as a black box, within which inputs are supplied to produce outputs, with there generally being a positive correlation between the two.
Traditional DEA models are only able to evaluate DMUs with deterministic inputs and outputs, while in real-world problems, data are usually uncertain. So far, various approaches have been introduced to overcome the uncertainty of data. In this paper, two robust DEA models are presented to evaluate the performance of systems with continuous uncertain data under constant return to scale (CRS) and variable return to scale (VRS) conditions. The main advantage of the proposed robust DEA models over the previous robust DEA models is that they are able to formulate uncertainty in both input and output data. Moreover, these models are also developed directly on basic traditional DEA models (not alternative models).

Research Question(s)
What is the status of effective indicators (citation and non-citation) in the ranking of valid scientific journals of psychology?
What are the top scientific journals in different fields of psychology?
What is the ranking status of psychology journals using the Data Envelopment Analysis (DEA) model?
What is the ranking status of psychology scientific journals based on the Anderson-Peterson CRS and Anderson-Peterson VRS models?
Literature Review

DEA is used for ranking journals in different fields. A study by Rosenthal and Weiss (2017) ranks business journals, using data from the Thomson Reuters Journal Citation Reports® (JCR). They ranked 358 business journals from five different JCR categories according to such citation-based factors as the number of articles, the number of citations, impact factor, five-year impact factor, immediacy index, eigenfactor score, and article influence score. They compute the relative efficiencies of the journals and thereby create plausible journal rankings that largely, but not completely, corroborate three widely used business publication journal ranking lists. In addition, we show how the different characteristics of the JCR data impact the DEA ranking model. Finally, they identify journals that are not on the business publication lists but consistently perform very well relative to those benchmark journals, and should possibly be included in the business publication ranking lists. They also identify journals whose inclusion in widely used business publication rankings cannot be justified by their methodology.

Methodology
In this study of data envelopment analysis technique, Anderson-Peterson model, CRS and VRS output-oriented, 2018 JCR ratings scientific journals of psychology were evaluated and the top journals ranked. This study has a quantitative approach and is applied research. The research method is scientometrics and bibliometrics using Operational Research techniques. In order to evaluate and rank psychology scientific journals, seven citation indicators were extracted from the Analytics Clarivate database. These indicators are the total number of citations, journal impact factor, five-year impact factor, immediate index, citeable items, journal influence factor, and journal article influence factor.
In this study, using data envelopment analysis technique, Anderson-Peterson model, CRS and VRS output-oriented, 2018 JCR ratings scientific journals of psychology were evaluated and the top journals ranked.

Results
Findings show that with the help of JCR indicators and using the Anderson-Peterson CRS and VRS model, more accurate rankings of journals can be achieved. The comparison of scientific journals is in the ranking of CRS and VRS models. The Journal of Psychological Science in the Public Interest tops in both models. The top 10 journals based on CRS respectively are: Psychological Science in the Public Interest, International Review of Sport and Exercise Psychology, Annual Review of Psychology, Psychological Bulletin, Psychological Inquiry, Behavioral and Brain Sciences, Social Issues and Policy Review, Personality and Social Psychology Review, Annual Review of Clinical Psychology, Journal of Personality and Social Psychology.The top 10 journals based on VRS respectively are: Psychological Science in the Public Interest, International Review of Sport and Exercise Psychology, Annual Review of Psychology, Frontiers in Psychology, Psychological Bulletin, Journal of Personality and Social Psychology, Trends in Cognitive Sciences, Psychological Science, Frontiers in Human Neurosciences.

Discussion
Here goes the text for your discussion.

Conclusion
Scientific journals of psychology were evaluated and ranked using the Anderson-Peterson CRS and VRS models. Also, the evaluation indicators of scientific journals in the Clarivate Analytics database were reviewed and the top three journals in 11 areas of psychology and the top 30 journals in this field were identified and introduced. The top 30 journals and their differences in CRS and VRS models were compared and analyzed. In the part of descriptive findings, the frequency of journals in different fields of psychology was found, including 10.25% of psychology journals in general, 10.91% of applied psychology journals, 1.86% of biological psychology journals, 17.17% scientific journals in the clinical psychology field, 10.11% scientific journals in advanced psychology field, 7.85% scientific journals in educational psychology field, 11.71% scientific journals in experimental psychology field, 1.73% scientific journals in mathematics psychology field, 18.28% are scientific journals in the field of interdisciplinary psychology, 1.73% are scientific journals in the field of psychoanalytic psychology, and 8.38% are scientific journals in the field of social psychology. Finally, considering the needs of the country's scientific community, conducting research with other research communities (different scientific fields and disciplines), conducting research and ranking journals with other indicators and normalized indicators are recommended.

Keywords

Main Subjects

اسماعیل‌­پور بندبنی، محمد، رمضانی پاکپور، فاطمه و رمضانی، ابوذر. (1395). بررسی شاخص‌های ارزیابی دو نظام رتبه‌بندی مجلات سالمندی. نشریه سالمندشناسی، ۱ (۲)، ۲۷-۳۵.
http://dx.doi.org/10.18869/acadpub.joge.1.2.27
زمانی، غلامحسین و عزیزی خالخیلی، طاهر. (1390). جایگاه مجلات علمی کشور در پایگاه‌های اطلاعاتی: موردمطالعه نشریات کشاورزی و منابع طبیعی. پژوهشنامه پردازش و مدیریت اطلاعات، 26 (4)، 803-823.
صالحی، کیوان و رحیمی، حسین. (1385). تبیین فرایند ارزیابی مجلات در موسسه اطلاعات علمی (آی.اس.آی). فصلنامه کتاب،17 (66)،141-160.
https://nastinfo.nlai.ir/article_534_286421a9859049224db516c579a46ce3.pdf
فضایلی، رضا. (1387). شاخص‌های ارزیابی مجلات و مقالات علمی از دیدگاه ISI. تولید علم، -(9)، 0-0. https://www.sid.ir/fa/journal/ViewPaper.aspx?id=160564.
فلاح‌زاده، علیرضا. (1397). سنجش همبستگی بین رتبه‌بندی بر مبنای سنجه‌های جایگزین نرمال‌شده و شاخص‌های استنادی IF، SJR و SNIP: مقایسه‌ای بین‌رشته‌ای در سطح مجلات علمی و پیشنهاد سنجه جایگزین ترکیبی. (پایان‌نامه کارشناسی ارشد، دانشکده علوم تربیتی و روان‌شناسی، دانشگاه شیراز)، شیراز.
کتابی، سعیده و شیخ ابومسعودی، علی. (1394). ارزیابی کارایی مجلات معتبر حوزه مدیریت بر اساس تکنیک تحلیل پوششی داده‌ها. مدیریت اطلاعات سلامت، 12(3)، 317-327.
https://doi.org/10.22122/him.v12i3.2159
محمد اسماعیل، صدیقه، ریاحی، عارف و صحبتی‌ها، فریبا. (1392). ارزیابی کمی و کیفی مجلات ایران در پایگاه استنادی اسکوپوس طی سال‌های 2000-2012. مجله علم‌‌‌­سنجی کاسپین،1 (1)، 33-39. http://dx.doi.org/10.22088/acadpub.BUMS.1.1.33
محمدنژاد عالی زمینی، یوسف (1391). شیوه‌­نامة پژوهش و نگارش علمی. تهران: پژوهشگاه علوم انسانی مطالعات فرهنگی.
معتمدی، فاطمه و رمضانی پاکپور، فاطمه. (1394). بررسی تطبیقی IF و SJR مجلات کتابداری و اطلاع‌­رسانی. علم‌­سنجی کاسپین، 2(1)، 50-56. http://dx.doi.org/10.22088/acadpub.BUMS.2.1.50
موسوی، چلک افشین، ریاحی، عارف و زارع، امین. (1397). ارزیابی کمی و کیفی و تعیین جایگاه مجلات نمایه‌شده‌ی حوزه‌ی پزشکی ایران در پایگاه اسکوپوس. پیاورد سلامت، ۱۲ (۱)، 11-24. https://civilica.com/doc/
مهرگان، محمد رضا. (1387). مدل­‌های کمّی در ارزیابی عملکرد سازمان­ها (تحلیل پوششی دادهها). تهران: انتشارات دانشکده مدیریت دانشگاه تهران.
یوسف پور، الهام. (1397). استفاده از تولاکنیک تحلیل پوششی داده­‌ها برای رتبه‌­بندی مؤسسات مالی اعتباری نور استان مازندران. (پایان‌نامه کارشناسی ارشد، موسسه آموزش عالی طبری)، بابل.
Andersen, P. & Petersen, N.C. (1993) A Procedure for Ranking Efficient Units in Data Envelopment Analysis. Management Science, 39(10), 1261-1264. https://doi.org/10.1287/mnsc.39.10.1261
Bou-Hamad, I., Anouze, A. L. & Larocque, D. (2017). An integrated approach of data envelopment analysis and boosted generalized linear mixed models for efficiency assessment. Annals of Operations Research, 253(1), 77-95. https://doi.org/10.1007/s10479-016-2348-4
Franceschini, F. & Maisano, D. (2011). Criticism on the hg-index. Scientometrics. 86(2), 339-346. https://doi.org/10.1007/s11192-010-0261-1
Garfield, E. (1972). Ciation analysis as a tool in journal evalution. Science. 178(4060), 471-479.  DOI: 10.1126/science.178.4060.471
Gross, P.L.K. & Gross, E.M. (1927). College libraries and chemical education. Science, 66(1713), 385-389.
https://doi.org/10.1126/science.66.1713.385
Kao, E. H., Hsu, C. H., Lu, Y. & Fung, H. G. (2016). Ranking of finance journals: A stochastic dominance analysis. Managerial Finance42(4), 312-323. https://doi.org/10.1108/MF-04-2015-0125
Lamp, J. W. (2009). Journal ranking and the dreams of academics. Online information review33(4), 827-830. https://doi.org/10.1108/14684520910985747
Mardani, A., Zavadskas, E. K., Streimikiene, D., Jusoh, A., & Khoshnoudi, M. (2017). A comprehensive review of data envelopment analysis (DEA) approach in energy efficiency. Renewable and Sustainable Energy Reviews, 70(c), 1298-1322. DOI: 10.1016/j.rser.2016.12.030
Martin, D. H., Kocher, G. & Sutter, M. (2000). Measuring Efficeincy of German Football Teams by DEA. Central European Journal of Operations Research and Economics, 12(3), 251-268.
McKinnon, A. C. (2013). Starry‐eyed: Journal rankings and the future of logistics research. International Journal of Physical Distribution & Logistics Management43(1), 6-17. https://doi.org/10.1108/09600031311293228
Nisonger, T.E. (1999). JASIS and library and information science journal rankings: A review and analysis of the last half-century. Journal of the American Society for Information Science, 50(11), 1004–1019. https://doi.org/10.1002/(SICI)1097-4571(1999)50:11%3C1004:AID-ASI6%3E3.0.CO;2-K
Ogawa, R. & Collom, E. (1998). Educational indicators: What are they? University of California, Riverside, CA: ERIC.
Pajić, D. (2015). On the stability of citation-based journal rankings. Journal of Informetrics9(4), 990-1006. https://doi.org/10.1016/j.joi.2015.08.005
Rosenthal, E. C. & Weiss, H. J. (2017). A data envelopment analysis approach for ranking journals. Omega, 70(c), 135-147. https://doi.org/10.1016/j.omega.2016.09.006
Walters, W. H. (2017). Composite journal rankings in library and information science: A factor analytic approach. The Journal of Academic Librarianship43(5), 434-442. https://doi.org/10.1016/j.acalib.2017.06.005
Zuccala, A., Guns, R., Cornacchia, R. & Bod, R. (2015). Can we rank scholarly book publishers? A bibliometric experiment with the field of history. Journal of the Association for Information Science and Technology, 66(7), 1333-1347. https://doi.org/10.1002/asi.23267
References [In Persian]
Fazayeli, R. (2008). Evaluation indexes of journals and scientific articles from the perspective of ISI. Production of science, -(9), 0-0. https://www.sid.ir/fa/journal/ViewPaper.aspx?id=160564. [In Persian]
Fallahzadeh, A. (2017). Measuring the correlation between ranking based on normalized alternative metrics and IF, SJR and SNIP citation indices: an interdisciplinary comparison at the level of scientific journals and the proposal of a combined alternative metric. (Master's Thesis, Department of information science and epistemology, Faculty of Education and Psychology, Shiraz university). Shiraz. [In Persian]
Ismail-Pour Bandbani, M., Ramezani Pakpour, F. & Ramezani, A. (2015). Examining the evaluation indices of two rating systems of geriatric journals. Journal of Gerontology, 1(2), 27-35. [In Persian]
Ketabi, S. & Sheikh Abu Masoudi, A. (2013). Evaluating the effectiveness of authoritative magazines in the field of management based on data coverage analysis technique. Health Information Management, 12(3), 317-327. [In Persian]
Mohammad Ismail, S., Riahi, A. & Sohbatiha, F. (2012). Quantitative and qualitative evaluation of Iranian journals in Scopus citation database during 2000-2012. Caspian Journal of Scientometrics, 1 (1), 33-39. [In Persian]
Mohammadnejad Aali Zamini, Y. (2011). Methodology of research and scientific writing. Tehran: Institute of Humanities and Cultural Studies. [In Persian]
Motamedi, F. & Ramazani Pakpour, F. (2014). Comparative study of IF and SJR of library and information journals. Caspian science, 2(1), 50-56. [In Persian]
Mousavi Chalak, A., Riahi, A. & Zare, A. (2017). Quantitative and qualitative evaluation and determining the position of indexed Iranian medical journals in the Scopus database. Journal of Payavard Salamat 12 (1), 11-24.https://civilica.com/doc/ [In Persian]
Mehregan, M. R. (2008). Quantitative models in evaluating the performance of organizations (coverage analysis of data) (second edition). Tehran: Tehran University School of Management Publications. [In Persian]
Salehi, K. & Rahimi, H. (2006). Explanation of the journal evaluation process in the Institute of Scientific Information (ISI). Book Quarterly, 17(66), 141-160. [In Persian]
Yusefpour, E. (2017). Using data envelopment analysis technique for ranking Noor credit financial institutions in Mazandaran province. (Master's thesis, Tabari Institute of Higher Education), Babol. [In Persian]
Zamani, G. H. & Azizi Khalkheili, T. (2011). Status of national scientific journals in databases: The case of agriculture and natural resources journals. Iranian Journal of Information Processing and Management, 26(4), 803-823. [In Persian].