Development and Validation of a Comprehensive Tool for Assessing Postgraduate Students’ Oral Presentations: Importance of the Role of Ergonomics
Main Article Content
Abstract
Introduction: The aim of this exploratory sequential mixed methods study was to develop an oral presentations assessment tool for postgraduate students’ oral presentations. Methods: First, a literature review and the ideas of 319 experienced professors were used to identify domains and potential items to develop the desired tool. Then the psychometric properties of the preliminary tool were measured using face and content validity, inter-rater agreement and test-retest reliability. Results: The qualitative phase indicated there should be four domains in the developed Oral Presentations Assessment Tool: subject knowledge, delivery, content and organization, and ergonomics, and that items in the developed tool should be weighted according to importance for the efficacy of a presentation. The final version included 19 items across four domains. Mean content validity index and content validity ratio scores were 0.93 and 0.76, respectively. Spearman’s rank correlation coefficient for the two evaluation periods was 0.92. The intra-class correlation coefficient was 0.78. Conclusion: The Oral Presentations Assessment Tool has appropriate psychometric properties and can be used as a valid and applicable instrument to assess postgraduate students’ oral presentations. Important cognitive factors in oral presentations in the form of an ergonomic domain was included for the first time, as part of this new comprehensive tool.
Downloads
Article Details
References
Khan A, Khan S, Zia-Ul-Islam S, Khan M. Communication Skills of a Teacher and Its Role in the Development of the Students’ Academic Success. J Educ Pract. 2017;8:18-21. doi: 10.1186/s40545-020-00217-3
McLaren I. Science Students’ Responses to an Oral Communication Skills Development Initiative: Attitude and Motivation. Int J Teach Learn High Educ. 2019;31:73-85.
Mercer-Mapstone LD, Matthews KE. Student perceptions of communication skills in undergraduate science at an Australian research-intensive university. Assess Eval High Educ. 2017;42:98-114. doi:10.1080/02602938.2015.1084492
Stuart L, Dahm E. 21st Century Skills for 21st Century Jobs. A Report of the US Department of Commerce. Washington, DC: U.S. Department of Education, U.S. Department of Labor, National Institute for Literacy and Small Business Administration 1999.
Van der Heiden P, Pohl C, Mansor SB, van Genderen J. The role of education and training in absorptive capacity of international technology transfer in the aerospace sector. Progress Aerospace Sci. 2015;76:42-54. doi:10.1016/j.paerosci.2015.05.003
Fletcher LB, Mullen LG, Stuart GJ. Assessment of Oral Presentations in an Accounting Program: Videotapes and Role Plays. The Accounting Educators’ Journal. 2019;28:193-214.
Adams K. Modelling success: enhancing international postgraduate research students’ self-efficacy for research seminar presentations. High Educ Res Develop. 2004;23:115-30. doi:10.1080/0729436042000206618
Taras M. Using assessment for learning and learning from assessment. Assess Eval High Educ. 2002;27:501-10. doi:10.1080/0260293022000020273
Liu W-I, Edwards H, Courtney M. Review of continuing professional education in case management for nurses. Nurse Educ Today. 2009;29:488-92. doi:10.1016/j.nedt.2008.11.004
Dent J, Harden R, Hunt D. A Practical Guide for Medical Teachers 5th Ed. Dundee: Elsevier; 2017.
Mokarami H, Jahangiri M, Javid AB, Ebrahimi MH, Zaroug Hossaini R, Barkhordari A, et al. Developing and validating tool for assessing the field internship course in the field of occupational health engineering. Iran Occup Health J. 2019;16:58-70.
Karwowski W. The Discipline of Human Factors and Ergonomics. Fourth Ed. Hoboken: Wiley; 2012.
Lehto MR, Landry SJ. Introduction to Human Factors and Ergonomics for Engineers. Boca Raton, Florida: Crc Press; 2012.
Rowley-Jolivet E. Image as Text. Aspects of the shared visual language of scientific conference participants. ASp la revue du GERAS. 2000;27-30:133-54. doi: 10.4000/asp.2093
Pearce L. How to Examine a Thesis. Berkshire, England: Open Unversity Press; 2005.
Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30:459-67. doi:10.1002/nur.20199
Lawshe CH. A quantitative approach to content validity. Pers Psychol. 1975;28:563-75. doi:10.1111/j.1744-6570.1975.tb01393.x
van Ginkel S, Gulikers J, Biemans H, Noroozi O, Roozen M, Bos T, et al. Fostering oral presentation competence through a virtual reality-based task for delivering feedback. Comput Educ. 2019;134:78-97. doi:10.1016/j.compedu.2019.02.006
Galindo C, Gregori P, Martínez V. Using videos to improve oral presentation skills in distance learning engineering master’s degrees. Int J Math Educ Sci Technol. 2020;51:103-14. doi:10.1080/0020739X.2019.1662118
Young C. Giving oral presentations. J Geog High Educ. 1998;22:263-8. doi:10.1080/03098269885958
Hill J, West H, Kneale P. Making the most of multi-disciplinary undergraduate research conferences. J Geog High Educ. 2018;42:311-6. doi:10.1080/03098265.2017.1402873
Alwi NFB, Sidhu GK. Oral presentation: Self-perceived competence and actual performance among UiTM business faculty students. Procedia Soc Behav Sci. 2013;90:98-106. doi:10.1016/j.sbspro.2013.07.070
Hartigan L, Mone F, Higgins M. How to prepare and deliver an effective oral presentation. BMJ. 2014;348:g2039. doi: 10.1136/bmj.g2039
Collins J. Education techniques for lifelong learning: giving a PowerPoint presentation: the art of communicating effectively. Radiographics. 2004;24:1185-92. doi:10.1148/rg.244035179
Shieh K-K, Lai Y-K. Effects of ambient illumination, luminance contrast, and stimulus type on subjective preference of VDT target and background color combinations. Percept Mot Skills.2008;107:336-52. doi:10.2466/pms.107.2.336-352
Anderson JR, Bower GH. Human Associative Memory. New York: Psychology Press; 2014.
Harolds JA. Tips for giving a memorable presentation, Part IV: Using and composing PowerPoint slides. Clin Nucl Med. 2012;37:977-80. doi:10.1097/RLU.0b013e3182614219.
Apperson JM, Laws EL, Scepansky JA. An assessment of student preferences for PowerPoint presentation structure in undergraduate courses. Comput Educ. 2008;50:148-53. doi:10.1016/j.compedu.2006.04.003
Craig RJ, Amernic JH. PowerPoint presentation technology and the dynamics of teaching. Innov High Educ. 2006;31:147-60. doi:10.1007/s10755-006-9017-5
Szabo A, Hastings N. Using IT in the undergraduate classroom: should we replace the blackboard with PowerPoint? Comput Educ. 2000;35:175-87. doi:10.1016/S0360-1315(00)00030-0