Assessment of the Psychometric Properties of E-learning Instructional Design Quality
With the ever-growing adoption of E-learning as an alternative mode for instructional delivery, and indeed as part of the strategic plan by higher learning institutions to foster open and distance learning, the development of empirically tested guidelines to evaluate E-learning instructional quality is timely. The purpose of the study was three-fold, that is to, explore the underlying structure of the E-learning instructional design quality construct, test the adequacy of its psychometric properties in terms of common method bias, reliability, convergent and discriminant validity, and cross validate the consistency of the measurement model across samples. The quantitative data was collected from a stratified random sample of 837 students undertaking CISCO E-learning courses at ten different institutions of higher learning in Uganda. A 38-item self-reported questionnaire to measure E-learners’ perceptions on E-learning instructional design quality served as the research instrument. The collected data were analysed using Exploratory Factor Analysis and Confirmatory Factor Analysis, with SPSS version 20.0 and AMOS version 22.0 softwares. The study results revealed that E-learning instructional design quality is a multidimensional construct with the sub dimensions of content quality, interface design quality, instructional strategies, content interactivity and E-learning feedback. Moreover, the measurement model was found to be free from common method bias and demonstrated adequacy in its validity and reliability. However, the results of cross validation indicated that the measurement model was not consistent across the three samples as shown by the variations in the model fit indices. The results are valuable to enable E-learning stakeholders to take strategic and evidence-based decisions regarding the integration of E-learning interventions for quality learning outcomes and enhanced future research in the domain of E-learning instructional design quality. Specifically, this study has successfully validated an E-learning instructional design quality questionnaire that educationists can use in evaluating E-learning courses regarding instructional design soundness.
Ally, M. (2004). Foundations of Educational Theory for online learning. In E. Terry, Anderson ; Fathi (Ed.), Theory and practice of online learning (pp. 3–32). Athabasca University.
Ambler, S. . (2000). User Interface Design: Tips and Techniques. Cambridge: Cambrige University Press.
Barnard, D., & Echolas, J. (2015). The Anatomy of K-12 Online Programs : Practical Ideas and Guidelines. London: Rowman and Littlefield.
Bonnel, W. (2008). Improving Feedback to Students in Online Courses. Nursing Education Perspectives, 29(5).
Casey, D. . (2008). The Historical Development of Distance Education through Technology. Tech Trends: Linking Research & Practice to Improve Learning, 52(2), 45–51.
Center for Disease Control and Prevention. (2013). CDC ’ s E-learning Essentials: A guide for creating quality Electronic learning. Antlanta: Centers for Disease Control and Prevention.
Clawson, S. (2007). Does quality matter? Measuring whether online course quality standards are predictive of student satisfaction in higher education. Capella University.
Comrey, A. L., & Lee, H. B. (1992). A first course in factor analysis (2nd ed.). New Jersey: Erlbaum Associates.
Debattista, M. (2018). A comprehensive rubric for instructional design in e-learning. International Journal of Information and Learning Technology, 35(2), 93–104. http://doi.org/10.1108/IJILT-09-2017-0092
Dick, W., Carey, L., & Carey, J. O. (2009). The Systematic Design of Instruction (7th ed.). New Jersey: Paerson.
Ekwue, U. E. (2013). Instructional strategies for online introductory college physics based on learning styles. Capella University. Retrieved from https://search.proquest.com/docview/1318534659?pq-origsite=gscholar
Endean, M., Bai, B., & Du, R. (2010). Quality standards in online distance education. International Journal of Continuing Education and Lifelong Learning, 3(1), 53–71. Retrieved from https://w5.hkuspace.hku.hk/journal/index.php/ijcell/article/view/109
Evans, C., & Sabry, K. (2003). Evaluation of the interactivity of Web-based learning systems: Principles and process. Innovations in Education and Teaching International, 40(1), 89–99. http://doi.org/10.1080/1355800032000038787
Faghih, B., Azadehfar, M. R., & Katebi, S. D. (2013). User Interface Design for E-Learning Software. The International Journal of Soft Computing and Software Engineering, 3(3), 786–794.
Galitz, W. O. (2007). The Essential Guide to An Introduction to GUI Design Principles and Techniques. Xtemp01.
Gaytan, J., & Mcewen, B. C. (2007). Effective Online Instructional and Assessment Strategies. The American Journal of Distance Education, 21(3), 117–132.
Guralnick, D. (2006). User Interface Design for Effective, Engaging E-Learning. In Proceedings of the International Conference on E-learning (pp. 22–23). Montreal: Kaleidoscope Learning.
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis (7th ed.). New York: Macmillan.
Hathaway, D. M. (2009). Assessing Quality Dimensions and Elements of Online Learning Enacted in a Higher Education Setting. Doctoral Thesis: George Mason University. George Mason University.
Hatziapostolou, T., & Paraskakis, I. (2010). Enhancing the Impact of Formative Feedback on Student Learning Through an Online Feedback System, 8(2), 111–122.
INACOL. (2011). National Standards for Quality Online Courses National Standards for Quality Online Courses (Version 2). Vienna: National Association for K-12 Online Learning.
Jackson, D. L. (2003). Revisiting Sample Size and Number of Parameter Estimates: Some Support for the N:q Hypothesis. Structural Equation Modeling, 10(1), 128–141.
Jung, I., & Latchem, C. (2007). Assuring quality in Asian open and distance learning. Open Learning: The Journal of Open and Distance Learning, 22(3), 235–250. http://doi.org/10.1080/02680510701619885
Jung, I., Wong, T. M., Li, C., Baigaltugs, S., Belawati, T., & Terbuka, U. (2011). Quality Assurance in Asian Distance Education: Diverse Approaches and Common Culture. International Review of Research in Open and Distance Learning, 12(2003), 63–83. Retrieved from http://0-eds.a.ebscohost.com.aupac.lib.athabascau.ca/eds/pdfviewer/pdfviewer?vid=11&sid=d1860e00-93df-4713-b29b-9b0e833cd54a%40sessionmgr4003&hid=4108
Khan, B. H. (2005). Managing E-Learning Strategies : Design, Deliver, Implementation and Evaluation. Hershey: Information Science Publishing.
Kline, R. (2016). Principles and Practice of Structural Equation Modeling (3rd ed.). New York: The Guilford Press.
Koslow, A. (2015). Relationship between students ’ perception of satisfaction in undergraduate online courses and course structure , interactions , learner autonomy and interface. Capella University.
Lorenzo, G. (2012). A Research Review about Online Learning: Are Students Satisfied? Why do Some Succeed and Others Fail? What Contributes to Higher Retention Rates and Positive Learning Outcomes? Internet Journal, 1(4), 46–55.
Marciniak, R. (2018). Quality Assurance for Online Higher Education Programmes : Design and Validation of an Integrative Assessment Model Applicable to Spanish Universities. International Review of Research in Open and Distance Learning, 19(2), 126–154. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/3443/4622
Martens, R. (1998). Does embedding support devices have an effect in independent learning ? European Journal of Open, Distance and E-Learning, 1–7.
Martens, R. (1993). Varying Embedded Support Devices in a Course: What is the Effect? Heerlen.
Martinez-Arguelles, M.-J., Plana, D., Hintzmann, C., Batalla-Busquets, J.-M., & Badia, M. (2015). Usefulness of Feedback in E-learning from the Students’ Perepsctive. Intangible Capital, 13(3), 627–645. http://doi.org/http://dx.doi.org/10.3926/ic.622 allowing
Masoumi, D. (2010). Quality in E-learning Within a Cultural Context. University of Gothenburg. University of Gothenburg.
Masoumi, D., & Lindstrom, B. (2012). Quality in e-learning : a framework for promoting and assuring quality in virtual institutions. Journal of Computer Assisted Learning, 28, 27–41. http://doi.org/10.1111/j.1365-2729.2011.00440.x
Matsunaga, M. (2011). How to factor-analyze your data right: Do’s, don’ts, and how-to’s. International Journal of Psychological Research, 3(1), 97–110.
Mhlanga, E., Krull, G., & Mallinson, B. (2013). Embedding Quality Improvement in Online Courses: A case study of seven African Universities. Johannesburg: SAIDE.
Murray, M., Perez, J., Geist, D., & Hedrick, A. (2013). Student interaction with content in online and hybrid courses: Leading horses to the proverbial water. Informing Science: The International Journal of an Emerging Transdiscipline, 16(1), 99–115.
Nordin, M. S., Ahmad, T. B. T., Zubairi, A. M., Ismail, N. A. H., Rahman, A. H. A., Trayek, F. A. A., & Ibrahim, M. B. (2016). Psychometric Properties of a Digital Citizenship Questionnaire. International Education Studies, 9(3), 71.
Pallant, J. (2007). Spss survival manual. Sydney: Allen & Unwin.
Petter, S., DeLone, W., & McLean, E. (2008). Measuring information systems success: Models, dimensions, measures, and interrelationships. European Journal of Information Systems, 17(3), 236–263. http://doi.org/https://doi.org/10.1057/ejis.2008.15
Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., & Podsakoff, N. P. (2003). Common Method Biases in Behavioral Research: A Critical Review of the Literature and Recommended Remedies. Journal of Applied Psychology, 88(5), 879–903. http://doi.org/10.1037/0021-9010.88.5.879
Quality Matters Program. (2013). Quality Matters Rubric Standards 2011 - 2013 edition.
Sahin, I., & Shelley, M. (2008). Considering Students ’ Perceptions : The Distance Education Student Satisfaction Model. Educational Technology & Society, 11(3), 216–223.
Simonson, M., Smaldino, S., Albright, M., & Zvacek, S. (2008). Foundations of distance education. Teaching and Learning at a Distance: Foundations of Distance Education (4th ed.).
Su, B., Bonk, C. J., Magjuka, R. J., Liu, X., & Lee, S. (2005). The importance of interaction in web-based education: A program-level case study of online MBA courses. Journal of Interactive Online Learning, 4(1), 1–19. http://doi.org/ISSN1541-4914
UNESCO. (2015). Leveraging Information and Communication Technologies to Achieve the Post-2015 Education Goal. Qingdao: UNESCO.
Vlachopoulos, D. (2016). Assuring quality in e-learning course design: The roadmap. International Review of Research in Open and Distance Learning, 17(6), 183–205. http://doi.org/10.19173/irrodl.v17i6.2784
Webb, A., & Moallem, M. (2016). Feedback and feed-forward for promoting problem-based learning in online learning environments. Malaysian Journal of Learning and Instruction, 13(2), 1–41.
Wixom, B. B. H., & Watson, H. J. (2001). An Empirical Investigation of The Factors Affecting. Data Warehousing Success. MIS Quarterly, 25Wixom, B(1), 17–41. http://doi.org/10.2307/3250957
Zhang, W., & Cheng, Y. L. (2012). Quality Assurance in E-Learning : PDPP Evaluation Model and its Application. International Review of Research in Open & Distance Learning, 13(3), 66–82.
Zimmerman, T. D. (2012). Exploring learner to content interaction as a success factor in online courses. International Review of Research in Open and Distance Learning, 13(4), 152–165.