تعیین حد نصاب قبولی راهکاری برای دستیابی به عدالت در ارزیابی: مطالعه‌ی مروری

نوع مقاله : Review Article

نویسندگان

1 دانشجوی کارشناسی ارشد، گروه آموزش پزشکی، مرکز مطالعات و توسعه آموزش پزشکی، دانشگاه علوم پزشکی اصفهان، اصفهان، ایران

2 استادیار، گروه آموزش پزشکی، مرکز مطالعات و توسعه آموزش پزشکی، دانشگاه علوم پزشکی اصفهان، اصفهان، ایران

چکیده

مقاله مروری




مقدمه: یکی از موضوعات چالش‌‌برانگیز در آموزش، برقراری عدالت در ارزیابی دانشجویان است. تعیین نمره‌ی قبولی ثابت برای همه‌ی آزمون‌ها، بدون در نظر گرفتن ویژگی‌های خاص هر آزمون و آزمون شونده، منصفانه نیست. برای برقراری عدالت، باید استاندارد قبولی هر آزمون بر اساس عواملی مانند سطح دشواری آزمون، توانایی دانشجویان، محیط یادگیری و اهداف آموزشی تعیین شود. این پژوهش با هدف ارائه‌ی راهکار‌هایی برای تعیین دقیق‌ حدنصاب نمره‌ی قبولی و بهبود کیفیت آزمون‌ها انجام شده است. انتظار می‌رود نتایج این مطالعه بتواند به مدیران برگزارکننده‌ی آزمون‌ها و اعضای هیأت علمی دانشگاه‌ها کمک کند تا فرایند ارزشیابی را بهبود بخشیده و در جهت برقراری عدالت آموزشی گامی مهم بردارند.
روش‌ها: این مطالعه از نوع مـروری روایتـی و جسـتجوی نظـام‌منـد بـوده است. مقالات چاپ شده فارسی و انگلیسی با کلید‌واژه‌های تعیین استاندارد (Standard setting)، نمره‌ی قبولی، حد نصاب قبولی (Minimum pass)، نقطه برش (Cut point) در پایگاه‌های اطلاعاتی نظیر PubMed, Google Scholar, Scopus, SID جستجو و در نهایت 37 مطالعه بر اساس معیارهای ورود و خروج وارد مطالعه شدند.
یافته‌ها: مقالات وارد شده در مطالعه، 16 روش تعیین حد نصاب قبولی را شرح داده بودند. به طور کلی این موارد در چهار دسته آزمون‌محور، آزمون‌شونده ‌محور، کارآیی محور و نتیجه محور تقسیم‌بندی می‌شوند.
نتیجه‌گیری: دانشگاه‌ها برای بهبود کیفیت آزمون‌های خود، با بهره‌گیری از متخصصین حوزه‌ی ارزیابی و امتحانات، اقدام به نظارت بر این فرایند نمایند و با استفاده از روش‌های ترکیبی و یا نقشه‌ی سؤال، آن را به عنوان یک رویکرد استاندارد در جهت دستیابی به عدالت در ارزیابی‌ها، مورد توجه قرار دهند.

تازه های تحقیق

عاطفه حیدری: Google Scholar, PubMed

شهرام شایان: Google Scholar, PubMed

 

کلیدواژه‌ها

موضوعات

عنوان مقاله [English]

Determining the Passing Score as A Strategy to Achieve Justice in Assessment: A Review Study

نویسندگان [English]

  • Atefeh Heidari 1
  • Shahram Shayan 2
1 MSc Student, Department of Medical Education, Center for Studies and Development of Medical Education, Isfahan University of Medical Sciences, Isfahan, Iran
2 Assistant Professor, Department of Medical Education, Center for Studies and Development of Medical Education, Isfahan University of Medical Sciences, Isfahan, Iran
چکیده [English]

Background: One of the challenging issues in education is establishing fairness in student evaluation. Determining a fixed passing score for all tests without considering the specific characteristics of each test and test taker is unfair. The passing standard for each test should be determined based on factors such as the level of difficulty of the test, the ability of the students, the learning environment, and educational goals to establish fairness. This study was conducted to provide solutions for accurately determining the passing score threshold and improving the quality of tests. It is expected that the results of this study can help test administrators and university faculty members to improve the evaluation process and take an influential step toward establishing educational fairness.
Methods: This study was a narrative review and systematic search. Published Persian and English articles were searched in databases such as PubMed, Google Scholar, Scopus, and SID using keywords including standard setting, passing score, minimum pass level, and cut point. Finally, 37 studies were included in the study based on the inclusion and exclusion criteria.
Findings: The articles included in the study described 16 methods for determining the passing score. These were broadly divided into four categories: test-centered, examinee-centered, performance-centered, and outcome-centered.
Conclusion: To improve the quality of their exams, universities should monitor this process by utilizing experts in the evaluation and using mixed methods or item mapping as a standard approach to achieving fairness in assessments.

کلیدواژه‌ها [English]

  • Educational measurement/methods
  • Social justice
  • Medical education

مراجع

  1. Mortaz Hejri S, Jalili M, Labaf A. Setting standard threshold scores for an objective structured clinical examination using Angoff method and assessing the impact of reality Chacking and discussion on actual scores [in Persian]. Iranian Journal of Medical Education 2012; 11(8): 885-94.
  2. Habibzadeh S, Delavar A, Farrokhi N, Minaei A, Jalili M. The use of rasch and item mapping in determining cut score of comprehensive pre internship exam [in Persian]. Res Med Edu 2019; 11(3): 59-70.
  3. Makarem A, Mahdavifard H, Gholami H. Evaluation of Passing Scores in Semiotics: An Objective Structured Clinical Examination for Medical Students of Mashhad University of Medical Sciences, Mashhad, Iran, 2015. Strides Dev Med Educ 2017; 14(1): e59227.
  4. Deputy Ministry of Health. The regulations of the supreme council of planning in medical sciences. Educational regulations of the non-continuous master's course [in Persian]. 2010. Available from: https://edu.mui.ac.ir/sites/edu/files/%D8%A2%DB%8C%DB%8C%D9%86%20%D9%86%D8%A7%D9%85%D9%87%20%DA%A9%D8%A7%D8%B1%D8%B4%D9%86%D8%A7%D8%B3%DB%8C%20%D8%A7%D8%B1%D8%B4%D8%AF.pdf.
  5. Maghsoodi F, YazdanFard M, Pouladi S, Mirzaei K. Using the methods of fixed score, angoff, yes/no angoff and three-level angoff to determine the standard and acceptance rate of nursing students participating in an objective structured clinical exam [in Persian]. Journal of Medical Education 2023; 23: 217-24.
  6. Grabovsky I, Pace J, Runyon CJAPM. Cut-score operating function extensions: penalty-based errors and uncertainty in standard settings. Appl Psychol Meas 2021; 45(7-8): 536-50.
  7. Barman A. Standard setting in student assessment: is a defensible method yet to come? Ann Acad Med Singap 2008; 37(11): 957-63.
  8. Sam AH, Millar KR, Westacott R, Melville CR, Brown CA. Standard setting Very Short Answer Questions (VSAQs) relative to Single Best Answer Questions (SBAQs): does having access to the answers make a difference? BMC Med Educ 2022; 22(1): 640.
  9. Torabinia N, Razavi SM, Shayan S, Hatamzade Z, Reisi M. Assessment of the Isfahan, Yazd, Kashan students' views about the dentistry basic science exam [in Persian]. J Iranian Journal of Medical Education 2019; 19(0): 216-24.
  10. Nemat Bakhsh M. Are the grades of basic science courses related to the grades of the same courses in the comprehensive exam of basic sciences? [in Persian]. Iranian Journal of Medical Education 2004; 4(1): 82-5.
  11. Daryazadeh S, Shayan S. Overview of the evolution process of student assessment methods in medical education: report of the change process in Iran based on the 8-step model [in Persian]. Iranian Journal of Medical Education 2017; 17: 32-42.
  12. Deputy Ministry of Health. Executive instructions for registration of the 70th course of the specialized Board exam. Available from: https://sanjeshp.ir/PDF.aspx?newsid=102501&type=application/pdf.
  13. Kareshki H, Hajiabadi F, Bagheri M, Moghaddam AG. A review of the basics of determining the cut-off point in academic achievement tests and the introduction of Angoff scientific method [in Persian]. Horizon of Medical Education Development 2021; 12(4): 85-96.
  14. Taylor DD, Reid C, Senhauser DA, Shively JA. Use of minimum pass levels on pathology examinations. J Med Educ 1971; 46(10): 876-81.
  15. Liu M, Liu KM. Descriptive terminology of education. Kaohsiung J Med Sci 2008; 24(12): 656-63.
  16. Yousefi HR. Descriptive terminology of education [in Persian]. Tehran, Iran: Kalrj Bartar Publications; 2011.
  17. Salzman DH, Rising KL, Cameron KA, Powell RE, Papanagnou D, Doty A, et al. Setting a minimum passing standard for the uncertainty communication checklist through patient and physician engagement. J Grad Med Educ 2020; 12(1): 58-65.
  18. Ward H, Chiavaroli N, Fraser J, Mansfield K, Starmer D, Surmon L, et al. Standard setting in Australian medical schools. BMC Med Educ 2018; 18(1): 80.
  19. Royal KD, Guskey TR. On the appropriateness of norm- and criterion-referenced assessments in medical education. Ear Nose Throat J 2015; 94(7): 252-4.
  20. Alshawwa L. Standard setting: a review of methods. Asian Journal of Education and Social Studies 2023; 42(2): 1-7.
  21. Kaufman DM, Mann KV, Muijtjens AM, van der Vleuten CPJAM. A comparison of standard-setting procedures for an OSCE in undergraduate medical education. Acad Med 2000; 75(3): 267-71.
  22. Park J, Ahn DS, Yim MK, Lee J. Comparison of standard-setting methods for the Korea Radiological technologist Licensing Examination: Angoff, Ebel, Bookmark, and Hofstee. J Educ Eval Health Prof 2018; 15: 32.
  23. Shayan SH. Using Patient Management Problem (EPMP) in Assessment of Clinical Competency. Iranian Journal of Medical Education 2011; 10(5): 1087-92.
  24. Dwyer T, Wright S, Kulasegaram KM, Theodoropoulos J, Chahal J, Wasserstein D, et al. How to set the bar in competency-based medical education: standard setting after an Objective Structured Clinical Examination (OSCE). BMC Med Educ 2016; 16: 1-7.
  25. Jalalizadeh M, Delavar A, Farokhi N, Askari M. Comparison of ANGOF-based IRT method and Bookmark method for standard Setting of MSRT language test. J Research in Teaching 2019; 7(4): 69-49.
  26. Yazdani S, Araghian F, Sanagoo A, Jooybari L. Hofstee an evaluation method in national medical science olympiad in Iran [in Persian]. Iranian Journal of Medical Education 2016; 16: 249-50.
  27. Jalili M, Khabaz Mafinejad M, Gandhamkar R, Mortazhejri S. Principles and evaluation methods of learners in medical sciences [in Persian]. Tehran: Academy of Medical Sciences of the Islamic Republic of Iran; 2017: 629.
  28. Taylor CA. Development of a modified Cohen method of standard setting. Med Teach 2011; 33(12): e678-e82.
  29. Zieky M, Perie M. A primer on setting cut scores on tests of educational achievement. [2006] Available from: https://www.ets.org/research/policy_research_reports/publications/publication/2006/dbkw.html.
  30. Bourque J, Skinner H, Dupré J, Bacchus M, Ainslie M, Ma IW, et al. Performance of the Ebel standard-setting method for the spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisting of multiple-choice questions. J Educ Eval Health Prof 2020;17: 12.
  31. Livingston SA, Kastrinos WJ. A study of the reliability of Nedelsky's method for choosing a passing score. ETS Research Report 1982; 1982(1): i-13.
  32. Shulruf B, Yang Y-Y, Huang P-H, Yang L-Y, Huang C-C, Huang C-C, et al. Standard setting made easy: validating the Equal Z-score (EZ) method for setting cut-score for clinical examinations. BMC Med Educ 2020; 20(1): 167.
  33. Aung MN, Jaroonvanichkul V, Deerojanawong J, Somboonwong J, Ishtiaq A, Wannakrairot PJ, et al. A New Method for Setting Standard in Medical Education, Applying Previous Year Cumulative GPA. 2019; 1(5).
  34. Anbari K, Ahmadi SAY, Rezaian S, Sabooteh T, Shirzadehgan R, Pirzadroozbahani TJ. Using Cochran formula for investigation of the significance of students' passing or failing: an evidence-based protocol in medical education. Iranian Journal of Medical Education 2018; 18: 319-21.
  35. Lahner F-M, Schauber S, Lörwald AC, Kropf R, Guttormsen S, Fischer MR, et al. Measurement precision at the cut score in medical multiple choice exams: Theory matters. Perspect Med Educ 2020; 9(4): 220-8.
  36. Kamali F, Shakour M, Yousefy A. Peer assessment in evaluation of medical sciences students [in Persian]. Iranian Journal of Medical Education 2012; 11(9): 1443-52.
  37. Sabouri M, Shayan S, Salehi A sabori mosih ss, salehi ahmad. Reviewing and reorganizing the educational process of the medical internship courses of Isfahan University of Medical Sciences in 2013 (the preliminary stage of determining the educational needs of the women's department). Iranian Journal of Education in Medical Sciences 2002; 8: 48-9.
  38. Kaftandjieva F. Methods for setting cut scores in criterion-referenced achievement tests. 2010; 170.
  39. Kazemi M, Shayan S, Akbari Jor H. Determining and improvement the effective factors of board examination in internal medicine–General surgery-Pediatrics & gynecology (GYN) discipline from perspective of stakeholders (faculty member, residents and educational manager) [in Persian]. Medicine and Spiritual Cultivation 2019; 28(4): 41-51.
مجله دانشکده پزشکی اصفهان
دوره 43، شماره 802
هفته 2، فروردین
فروردین و اردیبهشت 1404
صفحه 53-62

فایل ها

سابقه مقاله

  • تاریخ دریافت: 04 آذر 1403
  • تاریخ بازنگری: 27 آذر 1403
  • تاریخ پذیرش: 08 دی 1403

هم رسانی

ارجاع به این مقاله

آمار