Measuring The Effectiveness of University Programmes Based on Evaluation Models: A Meta-Analysis
DOI:
https://doi.org/10.33102/abqari.vol20no2.206Keywords:
evaluation, model, effectiveness, programmeAbstract
This article presents an extensive systematic review and meta-analysis on the effectiveness of university programmes based on different evaluation models. This study was conducted on primary sources published between year 2000 to 2018 using systematic searches from online databases including EBSCOhost, CENGAGE Learning, Cochrane Library, Clinical Key, ProQuest, ScienceDirect, and Scopus. There were, twenty-eight studies found on measuring the effectiveness of university programmes using different evaluation models. From these studies, the most common evaluation models used are the Context Input Process Product (CIPP) model and the Kirkpatrick model. It is concluded that the effectiveness of a programme needs to be improved through the implementation of optimising training designs, redefining training roles, management commitment, attention to individual and use of ongoing evaluation.
Downloads
References
Alkin, M. C. (1969). Evaluating theory development. Evaluation Comment, 2, 2-7.
Azizi, Y. (2001). The using of model context, input, process and products (CIPP) in learning programmes assessment. Paper presented at the International Conference on Challenges and Prospects in Teacher Education. University of Technology Malaysia.
Barr. H & Ross,F. (2006). Mainstreaming interprofessional education in the United Kingdom: A position paper. Journal of Interprofessional Care, 20(2):96-104.
Billings, D.M., & Halstead, J. A. (2015). Teaching in nursing: A guide for faculty. (5th ed.). St. Louis, Missouri: Saunders.
Briggs, D., & Domingue, B. (2011). Due diligence and the evaluation of teachers: A review of the value-added analysis underlying the effectiveness rankings of Los Angeles Unified School District teachers by the Los Angeles Times. Boulder, CO: National Education Policy Centre.
Brinkerhoff, R. O., Brethower, D. M., Hluchyj, T., & Nowakowski, J. R. (1983). Programme evaluation: A practitioner's guide for trainers and educators. The Netherlands: Kluwer Academic Publishers Group.
Brinkerhoff, R. O. (1989). Using evaluation to transform training. Special Issue: Evaluating Training Programmes in Business and Industry, 44, 5-20.
Cambridge Dictionary. (2018). Retreived from https://dictionary.cambridge.org/dictionary/english/effective
Commission on Collegiate Nursing Education. (2013). Standards for accreditation of baccalaureate and graduate nursing programmes. Retrieved from http://www.aacn.nche.edu/ccne-accreditation/Standards-Amended-2013.pdf.
Creswell, J.W. (2003). Research design qualitative, quantitative and mixed method approaches. New York: SAGE Publications.
Crombie, I.K. (1996). The Pocket Guide to Appraisal. London:BMJ Publishing Group .
Cronbach, L. J. (1963). Course improvement through evaluation. Teacher College Record, 64, 672-683.
Davis, L.M., Robert,B., Jennifer, L.S., Jessica, S., & Jeremy, N. V. M. (2013). Evaluating the effectiveness of correctional education: A meta-analysis of programs that provide 12 education to incarcerated adults. Santa Monica, CA: RAND Corporation retrieved from https://www.rand.org/pubs/research_reports/RR266.html. Also available in print form.
Elsevier, B.V. (2008). Scopus overview: What is it? [Online]. Available: http://www.info.scopus.com/about/.
Fink, A. (2005). Conducting research literature reviews: From the internet to paper. New York: Sage Publication Inc.
Fitzpatrick, J. L., Saners, J. R., & Worthen, B. R. (2004). Program evaluation: Alternatives, approaches, and practical guidelines. (3rd ed.). Boston: Pearson Education Inc.
Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5(10),3-8.
Greene,J.P.(1998). A meta-analysis of the effectiveness of bilingual education. Claremont, CA: Thomas Rivera Policy Institute.
Greenhalgh, T. (2000). How to read a paper: The basics of evidence-based medicine. (6th ed.) London :BMJ Publishing Group.
Griffin, P., & Nix, P. (1991). Educational assessment and reporting. Sydney: Harcout Brace Javanovich Publisher.
Khan,K.S., Kunz, R., Kleijnen,J., & Antes,G. (2003). Systematics reviews to support evidence-based medicine. London: The Royal Society of Medicine Press Ltd.
Kirkpatrick, D.L. (1998). Evaluating training programmes: The four levels. San Francisco: Berretp-Koehler Publisher, Inc.
Kitchenham, B.A., & Charters, S. (2007). Procedures for performing systematic literature reviews in software engineering, EBSE Tech. Report Version 2.3, EBSE-2007-01, Software Engineer, Group, Keele University of Durham, UK.
Laura Keeler, R., Lee, K., Kim, P., Katie, D. M., & Jill, A. H. (2018). Effectiveness of the wheelchair skills training program: A systematic review and meta-analysis. Disability and Rehabilitation. Assistive Technology Journal,14(4),391-409. DOI: 10.1080/17483107.2018.1456566.
Leedly, P.D., & Ormrod, J.E. (2005). Practical research planning and design. (8th ed.). United Kingdom: Pearson Merril Prentice Hall.
Leverenz, L. (2009). Allied health education program accreditation - what does it mean? Retrieved from http://www.studyoverseas.com/alliedhealthcare/ articles/healthed.html.
Mardapi, D. (2000). Evaluasi pendidikan. In a paper presented at the National Education Convention. Jakarta: Jakarta State University, 19-23 September
McNamara, C. (1998). Basic Guide to Program Evaluation. https://www.google.com.my/search?q=McNamara%2C+C.+(1998).-+Basic+Guide+to+program+evaluation.
Megan, L., & Patricia, C. (2017). Using the CIPP model to assess nursing education program quality and merit. Teaching and learning in nursing xxx (2017) xxx–xxx. (Article available in sciencedirect.com).
Mendes, E. (2005). A systematic review of web engineering research. In International Symposium on Empirical Software Engineering. 498-507.
Narges,N., & Mohammad Arab. (2016). Evaluation of selected faculties at Tehran University of medical sciences using CIPP model in students and graduates point of view. Evaluation and Program Planning http://dx.doi.org/10.1016/j.evalprogplan.2016.06.013
Newton, X., Darling-Hammond, L., Haertel, E., & Thomas, E. (2010). Value-added modelling of teacher effectiveness: An exploration of stability across models and contexts. Educational Policy Analysis Archives, 18(23), 1-27.
Nurfaradilla,N., Siti Norhidayah, R., Mohammad Iskandar, S., Kasmah, A.B., Lilia, H.,& Sharifah Nor, P.(2010). Effectiveness of school wide enrichment model (MPSS) in Mara Junior Science College. Procedia Social and Behavioral Sciences, 7(C), 43-49.
Pazargadi, M., & Azadi Ahmadabadi, G. (2008) Quality and quality assessment in universities and higher education institutions . Tehran: Boshra.
Petticrew, M., & Roberts,H. (2006). Systematic reviews in the social sciences a practical guide. United States: Blackwell Publishing.
Pignon, J., & Poynard, T. (1993). [Meta-analysis of therapeutic trials. Principles, methods and critical reading]. La Revue du praticien, 43(18): 2383-2386.
Rothstein, J. (2007). Do value-added models add value? Tracking, fixed effects, and causal inference. CEPS Working Paper No. 159. Cambridge, MA: National Bureau of Economic Research.
Sopoaga, F., Kokaua, J., Van der Meer, J., Lameta-Huro, Malia., Zaharic, Tony., Richards, Rose., & Inder, Marie. (2017). Evaluating the effectiveness of a programme for improving the participation and academic success of an underrepresented minority group in New Zealand. Evaluation and Program Planning http://dx.doi.org/10.1016/j.evalprogplan.2017.06.002.
Spencer,L., Ritchie,J., Lewis,J., & Dillon,L.(2003). Quality n qualitative evaluation: A framework for assessing research evidence. Government Chief Social Researchers’ Office. London.
Stanley, T. D., & Jarrell, S. B. (1989). Meta‐Regression analysis: A quantitative method of literature surveys. Journal of Economic Surveys, 3(2), 161-170.
Stark, J.S., & Thomas, A. (1994). Assessment and programme evaluation. Needham Heights: Simon & Schuster Custom Publishing.
Stufflebeam, D. L., Foley, W. J., Gephart, W. J., Guba, E. G., Hammond, H. D., Merriman, H. O., & Provus,M.M. (1971). Educational evaluation and decision making. Itasca, IL: Peacock.
Suchman, E. A. (1967). Evaluative research. New York: Russel Sage Foundation Talmage.
Syamsu, Q.B. (2013). The implementation of kirkpatrick's evaluation model in the learning of initial value and boundary condition problems. International Journal of Learning & Development, 3(5), 74-88.
Tyler, R.W. (1945). General statement on evaluation. Journal of Educational Research, 35, 492-501.
Tyler, R. W. (1950). Basic principles of curriculum and instruction. Chicago: University of Chicago Press. Wallace.
Willig, A.C. (1985). A meta-analysis of selected studies on the effectiveness of bilingual education. Review of Educational Research, 55(3), 269-318.
Wohlin, C., Runeson, P., Host, M., Ohlsson,M.C., Regnell, B., & Wesslen.A. (2000). Experimentation in software engineering: An introduction. Boston: Kluwer Academic Publisher.
Downloads
Published
How to Cite
Issue
Section
License
The copyright of this article will be vested to author(s) and granted the journal right of first publication with the work simultaneously licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license, unless otherwise stated.
