Journal of Research in Education Sciences - Journal directory listing - Volume 62 (2017) - Journal of Research in Education Sciences【62(2)】June (Special Issue: Digital Learning: Retrospect and Prospect)

Search by article title, author,
keywords, and DOI

CONTRIBUTORS
NOTICE

[Guide for Submission]

[Style for Thesis Writing]

[Review of Research Papers]

[The Copyright]

[Flow of Review System]

[Regulations Governing]

[Editorial Board]

[Advisory Board]

[Form Download]

Call for papers for the special issue “Prevention and Treatment of Addictive Behaviors on Campus” (Final submission deadline: 2021/8/31) The Journal of Research in Education Sciences will no longer provide printed copies Call for papers for the special issue “Educational Research Related to Southeast Asian Countries” (Final submission deadline: 2018/12/31)

Journal directory listing - Volume 62 (2017) - Journal of Research in Education Sciences【62(2)】June (Special Issue: Digital Learning: Retrospect and Prospect)

Back

(Special Issue) Quality Assessment and Situational Analysis of Experimental E-Learning Designs:A Case Study of Mobile Learning Author: Han-Yueh Lee(Research Center for Psychological and Educational Testing, National Taiwan Normal University), Je-Ming Yang(Research Center for Psychological and Educational Testing, National Taiwan Normal University), Yao-Ting Sung(Department of Educational Psychology and Counseling, National Taiwan Normal University)

Vol.&No.：Vol. 62, No.2
Date：June 2017
Pages：31-60
DOI：10.6209/JORIES.2017.62(2).02

Abstract：

Because of the substantial development of mobile devices and educational software in recent years, the results of mobile learning–based interventions represent a popular research topic for investigation. Although experiment quality is the basis of empirical research, few studies have explored this issue. Thus, the present study investigated the shortfalls in existing experimental research designs related to mobile learning over the past decade and offers suggestions in this paper. The researchers collected data from all 197 experimental studies on mobile learning published in the Education Resources Information Center and Institute of Science Index from 2003 to 2013. The findings of the present study are described as follows: (1) Quasi-experimental designs represent the most frequently used design type (61%); however, among the quasi-experimental studies, 25% did not consider baseline equivalence. (2) Over half of the studies may not have met basic statistical assumptions, and approximately 70% used insufficient sample sizes, leading to low statistical power and imprecise effect size estimation. (3) Half of the studies did not provide information on test reliability and validity. Finally, this paper discusses the results and their implications for future research and practice.

Keywords：experimental design, mobile learning, research quality

《Full Text》

References：

張菀珍、葉榮木（2014）。應用多媒體行動學習系統輔助大學生情緒管理學習成效與評量之探究。教育科學研究期刊，59（4），99-136。doi:10.6209/JORIES.2014.59(4).04【Chang, W.-J., & Yeh, Z.-M. (2014). Exploration of learning effectiveness and assessment of emotion management for college students by using a multimedia mobile learning system. Journal of Research in Education Sciences, 59(4), 99-136. doi:10.6209/JORIES.2014.59(4).04】
Ardito, C., Costabile, M. F., de Angeli, A., & Lanzilotti, R. (2012). Enriching archaeological parks with contextual sounds and mobile technology. ACM Transactions on Computer-Human Interaction, 19(4), 1-30. doi:10.1145/2395131.2395136
Aron, A., & Aron, E. N. (1999). Statistics for psychology (2nd ed.). Upper Saddle River, NJ: Prentice-Hall.
Basoglu, E. B., & Akdemir, O. (2010). A comparison of undergraduate students English vocabulary learning: Using mobile phones and flash cards. Turkish Online Journal of Educational Technology, 9(3), 1-7. Retrieved from http://www.tojet.net
Bebell, D., & O’Dwyer, L. M. (2010). Educational outcomes and research from 1:1 computing settings. Journal of Technology, Learning, and Assessment, 9(1), 5-15. Retrieved from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1606

» More

張菀珍、葉榮木（2014）。應用多媒體行動學習系統輔助大學生情緒管理學習成效與評量之探究。教育科學研究期刊，59（4），99-136。doi:10.6209/JORIES.2014.59(4).04【Chang, W.-J., & Yeh, Z.-M. (2014). Exploration of learning effectiveness and assessment of emotion management for college students by using a multimedia mobile learning system. Journal of Research in Education Sciences, 59(4), 99-136. doi:10.6209/JORIES.2014.59(4).04】
Ardito, C., Costabile, M. F., de Angeli, A., & Lanzilotti, R. (2012). Enriching archaeological parks with contextual sounds and mobile technology. ACM Transactions on Computer-Human Interaction, 19(4), 1-30. doi:10.1145/2395131.2395136
Aron, A., & Aron, E. N. (1999). Statistics for psychology (2nd ed.). Upper Saddle River, NJ: Prentice-Hall.
Basoglu, E. B., & Akdemir, O. (2010). A comparison of undergraduate students English vocabulary learning: Using mobile phones and flash cards. Turkish Online Journal of Educational Technology, 9(3), 1-7. Retrieved from http://www.tojet.net
Bebell, D., & O’Dwyer, L. M. (2010). Educational outcomes and research from 1:1 computing settings. Journal of Technology, Learning, and Assessment, 9(1), 5-15. Retrieved from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1606
Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., & Abrami, P. C. (2014). A meta-analysis of blended learning and technology use in higher education: From the general to the applied. Journal of Computing in Higher Education, 26(1), 87-122. doi:10.1007/s12528　-013-9077-3
Billings, E. S., & Mathison, C. (2012). I get to use an iPod in school? Using technology-based advance organizers to support the academic success of English learners. Journal of Science Education and Technology, 21(4), 494-503. doi:10.1007/s10956-011-9341-0
Brooks, G., Miles, J. N. V., Torgerson, C. J., & Torgerson, D. J. (2006). Is an intervention using computer software effective in literacy learning? A randomised controlled trial. Educational Studies, 32(2), 133-143. doi:10.1080/03055690500416116
Burston, J. (2015). Twenty years of MALL project implementation: A meta-analysis of learning outcomes. ReCALL, 27(1), 4-20. doi:10.1017/S0958344014000159
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multritrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105. doi:10.1037/h0046016
Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research. Chicago, IL: Rand McNally.
Chao, P.-Y., & Chen, G.-D. (2009). Augmenting paper-based learning with mobile phones. Interacting with Computers, 21(3), 173-185. doi:10.1016/j.intcom.2009.01.001
Chen, N.-S., Hsieh, S.-W., & Kinshuk. (2008). Effects of short-term memory and content representation type on mobile language learning. Language Learning & Technology, 12(3), 93-113. Retrieved from http://llt.msu.edu/vol12num3/chenetal.pdf
Chen, Y.-S., Kao, T.-C., & Sheu, J.-P. (2005). Realizing outdoor independent learning with a butterfly-watching mobile learning system. Journal of Educational Computing Research, 33(4), 395-417. doi:10.2190/0PAB-HRN9-PJ9K-DY0C
Cheung, A. C.-K., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis. Educational Research Review, 9, 88-113. doi:10.1016/j.edurev.2013.01.001
Chu, H.-C., Hwang, G.-J., Huang, S.-X., & Wu, T.-T. (2008). A knowledge engineering approach to developing e-libraries for mobile learning. Electronic Library, 26(3), 303-317. doi:10.1108/026 40470810879464
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49(12), 997-1003. doi:10. 1037/0003-066X.49.12.997
D’Alonzo, K. T. (2004). The Johnson-Neyman procedure as an alternative to ANCOVA. West Journal of Nursing Research, 26(7), 804-812. doi:10.1177/0193945904266733
Dupont, W. D., & Plummer, W. D. (1990). Power and sample size calculations: A review and computer program. Controlled Clinical Trials, 11(2), 116-128. doi:10.1016/0197-2456(90)90005-M
Edwards, C. M., Rule, A. C., & Boody, R. M. (2013). Comparison of face-to-face online mathematics learning of sixth graders. Journal of Computers in Mathematics and Science Teaching, 32(1), 25-47. Retrieved from https://www.learntechlib.org/p/39231
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175-191. doi:10.3758/BF03193146
Ferguson, G. A., & Takane, Y. (1989). Statistical analysis in psychology and education (6th ed.). New York, NY: McGraw-Hill.
Fleischer, H. (2012). What is our current understanding of one-to-one computer projects: A systematic narrative research review. Educational Research Review, 7(2), 107-122. doi:10.1016/ j.edurev.2011.11.004
Fraenkel, J. R., Wallen, N. E., & Hyun, H. (2011). How to design and evaluate research in education. New York, NY: McGraw-Hill Education.
Frohberg, D., Goth, C., & Schwabe, G. (2009). Mobile learning projects– A critical analysis of the state of the art. Journal of Computer Assisted Learning, 25(4), 307-331. doi:10.1111/j.1365- 2729.2009.00315.x
Gentry, T. (2008). PDAs as cognitive aids for people with multiple sclerosis. American Journal of Occupational Therapy, 62(1), 18-27. doi:10.5014/ajot.62.1.18
Glass, G. V., Peckham, P. D., & Sanders, J. R. (1972). Consequences of failure to meet assumptions underlying the fixed effects analyses of variance and covariance. Review of Educational Research, 42(3), 237-288. doi:10.2307/1169991
Hedman, L., & Sharafi, P. (2004). Early use of internet-based educational resources: Effects on students’ engagement modes and flow experience. Behaviour & Information Technology, 23(2), 137-146. doi:10.1080/01449290310001648251
Howell, D. C. (2002). Statistical methods for psychology (5th ed.). Pacific Grove, CA: Duxbury.
Hsi, S. (2003). A study of user experiences mediated by nomadic web content in a museum. Journal of Computer Assisted Learning, 19(3), 308-319. doi:10.1046/j.0266-4909.2003.jca_023.x
Hsu, Y.-C., & Ching, Y.-H. (2013). Mobile computer-supported collaborative learning: A review of experimental research. British Journal of Educational Technology, 44(5), E111-E114. doi:10. 1111/bjet.12002
Huck, S. W. (2008). Reading statistics and research (5th ed.). Boston, MA: Pearson Education.
Hung, J.-L., & Zhang, K. (2012). Examining mobile learning trends 2003-2008: A categorical meta-trend analysis using text mining techniques. Journal of Computer Higher Education, 24(1), 1-17. doi:10.1007/s12528-011-9044-9
Hwang, G.-J., Shi, Y.-R., & Chu, H.-C. (2011). A concept map approach to developing collaborative mindtools for context-aware ubiquitous learning. British Journal of Educational Technology, 42(5), 778-789. doi:10.1111/j.1467-8535.2010.01102.x
Hwang, G.-J., & Tsai, C.-C. (2011). Research trends in mobile and ubiquitous learning: A review of publications in selected journals from 2001 to 2010. British Journal of Educational Technology, 42(4), E65-E70. doi:10.1111/j.1467-8535.2011.01183.x
Keppel, G. (1991). Design and analysis: A researcher’s handbook (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.
Kerawalla, L., O’Connor, J., Underwood, J., duBoulay, B., Holmberg, J., Luckin, R., …Tunley, H. (2007). Exploring the potential of the homework system and tablet PCs to support continuity of numeracy practices between home and primary school. Educational Media International, 44(4), 289-303. doi:10.1080/09523980701680904
Kert, S. B. (2011). The use of SMS support in programming education. Turkish Online Journal of Educational Technology, 10(2), 268-273. Retrieved from http://files.eric.ed.gov/fulltext/ EJ932245.pdf
Ketamo, H. (2003). An adaptive geometry game for handheld devices. Educational Technology & Society, 6(1), 83-95. Retrieved from http://ifets.info/journals/6_1/ketamo.pdf
Kirk, R. E. (1995). Experimental design procedures for the behavioral sciences (3rd ed.). Pacific Grove, CA: Brooks/Cole.
Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56(5), 746-759. doi:10.1177/0013164496056005002
Kondo, M., Ishikawa, Y., Smith, C., Sakamoto, K., Shimomura, H., & Wada, N. (2012). Mobile assisted language learning in university EFL courses in Japan: Developing attitudes and skills for self-regulated learning. ReCALL, 24(2), 169-187. doi:10.1017/S0958344012000055
Kulik, C.-L. C., & Kulik, J. A. (1991). Effectiveness of computer-based instruction: An updated analysis. Computers in Human Behavior, 7(1-2), 75-94. doi:10.1016/0747-5632(91)90030-5
Liao, C. Y.-K. (2007). Effects of computer-assisted instruction on students’ achievement in Taiwan: A meta-analysis. Computers & Education, 48(2), 216-233. doi:10.1016/j.compedu.2004.12.005
Lin, T.-J., Duh, H. B.-L., Li, N., Wang, H.-Y., & Tsai, C.-C. (2013). An investigation of learners’ collaborative knowledge construction performances and behavior patterns in an augmented reality simulation system. Computers & Education, 68, 314-321. doi:10.1016/j.compedu.2013. 05.011
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
Littell, J. H., Corcoran, J., & Pillai, V. (2008). Systematic reviews and meta-analysis. New York, NY: Oxford University press.
Liu, C.-C., Chou, C.-C., Liu, B.-J., & Yang, J.-W. (2006). Improving mathematics teaching and learning experiences for hard of hearing students with wireless technology-enhanced classrooms. American Annals of the Deaf, 151(3), 345-355. doi:10.1353/aad.2006.0035
Liu, T.-Y., & Chu, Y.-L. (2010). Using ubiquitous games in an English listening and speaking course: Impact on learning outcomes and motivation. Computers & Education, 55(2), 630-643. doi:10. 1016/j.compedu.2010.02.023
Liu, C.-C., & Hong, Y.-C. (2007). Providing hearing-impaired students with learning care after classes through smart phones and the GPRS network. British Journal of Educational Technology, 38(4), 727-741. doi:10.1111/j.1467-8535.2006.00656.x
Liu, T.-C., Lin, Y.-C., & Paas, F. (2014). Effects of prior knowledge on learning from different compositions of representations in a mobile learning environment. Computers & Education, 72, 328-338. doi:10.1016/j.compedu.2013.10.019
Looi, C.-K., Zhang, B., Chen, W., Seow, P., Chia, G., Norris, C., & Soloway, E. (2011). 1:1 mobile inquiry learning experience for primary science students: A study of learning effectiveness. Journal of Computer Assisted Learning, 27(3), 269-287. doi:10.1111/j.1365-2729.2010.00390.x
Mayr, S., Erdfelder, E., Buchner, A., & Faul, F. (2007). A short tutorial of GPower. Tutorials in Quantitative Methods for Psychology, 3(2), 51-59. doi:10.20982/tqmp.03.2.p051
Morris, N. P., Ramsay, L., & Chauhan, V. (2012). Can a tablet device alter undergraduate science students’ study behavior and use of technology? Advances in Physiology Education, 36(2), 97-107. doi:10.1152/advan.00104.2011
Oberg, A., & Daniels, P. (2013). Analysis of the effect a student-centred mobile learning instructional method has on language acquisition. Computer Assisted Language Learning, 26(2), 177-196. doi:10.1080/09588221.2011.649484
Pagano, R. R. (2007). Understanding statistics in the behavioral sciences (8th ed.). Belmont, CA: Thomson Higher Education.
Penuel, W. R. (2006). Implementation and effects of one to one computing initiatives: A research synthesis. Journal of Research on Technology in Education, 38(3), 329-348. doi:10.1080/ 15391523.2006. 10782463
Rockinson-Szapkiw, A. J., Courduff, J., Carter, K., & Bennett, D. (2013). Electronic versus traditional print textbooks: A comparison study on the influence of university students’ learning. Computers & Education, 63, 259-266. doi:10.1016/j.compedu.2012.11.022
Roschelle, J., Rafanan, K., Bhanot, R., Estrella, G., Penuell, B., Nussbaum, M., & Claro, S. (2010). Scaffolding group explanation and feedback with handheld technology: Impact on students’ mathmatics learning. Educational Technology Research and Development, 58(4), 399-419. doi:10.1007/s11423-009-9142-9
Sandberg, J., Maris, M., & de Geus, K. (2011). Mobile English learning: An evidence-based study with fifth graders. Computers & Education, 57(1), 1334-1347. doi:10.1016/j.compedu.2011.01.015
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton-Mifflin.
Sink, C. A., & Mvududu, N. H. (2010). Statistical power, sampling, and effect sizes: Three keys to research relevancy. Counseling Outcome Research and Evaluation, 1(2), 1-18. doi:10.1177/ 2150137810373613
Slavin, R. E. (2002). Evidence-based education policies: Transforming educational practice and research. Educational Researcher, 31(7), 15-21. doi:10.3102/0013189X031007015
Slavin, R. E. (2003). A reader’s guide to scientifically based research. Educational Leadership, 60(5), 12-16. Retrieved from http://www.ascd.org/publications/educational-leadership/feb03/vol60/ num05/A-Reader’s-Guide-to-Scientifically-Based-Research.aspx
Slavin, R., & Madden, N. A. (2011). Measures inherent to treatments in program effectiveness reviews. Journal of Research on Educational Effectiveness, 4(4), 370-380. doi:10.1080/19345747. 2011.558986
Solhaug. (2009). Two conﬁgurations for accessing classroom computers: Differential impact on students’ critical reﬂections and their empowerment. Journal of Computer Assisted Learning, 25(5), 411-422. doi:10.1111/j.1365-2729.2009.00318.x
Sung, Y.-T., Chang, K.-E., Lee, Y.-H., & Yu, W.-C. (2008). Effects of a mobile electronic guidebook on visitors’ attention and visiting behaviors. Educational Technology & Society, 11(2), 67-80. Retrieved from http://www.ifets.info/journals/11_2/7.pdf
Sung, Y.-T., Chang, K.-E., & Liu, T.-C. (2016). The effects of integrating mobile devices with teaching and learning on students’ learning performance: A meta-analysis and research synthesis. Computers & Education, 94, 252-275. doi:10.1016/j.compedu.2015.11.008
Sung, Y.-T., Chang, K.-E., & Yang, J.-M. (2015). How effective are mobile devices for language learning? A meta-analysis. Educational Research Review, 16, 68-84. doi:10.1016/j.edurev.2015. 09.001
Tabachnick, B., & Fidell, L. (2007). Using multivariate statistics. Boston, MA: Pearson/Allyn & Bacon.
Tan, T.-H., Liu, T.-Y., & Chang, C.-C. (2007). Development and evaluation of an RFID-based ubiquitous learning environment for outdoor learning. Interactive Learning Environments, 15(3), 253-269. doi:10.1080/10494820701281431
U.S. Department of Education. (2002). Strategic plan 2002-2007. Retrieved from http://www2. ed.gov/about/reports/strat/plan2002-07/index.html
Valentine, J. C., & Cooper, H. (2008). A systematic and transparent approach for assessing the methodological quality of intervention effectiveness research: The study design and implementation assessment device (Study DIAD). Psychological Methods, 13(2), 130-149. doi: 10.1037/1082-989X.13.2.130
Valentine, J. C., & McHugh, C. M. (2007). The effects of attrition on baseline comparability in randomized experiments in education: A meta-analysis. Psychological Methods, 12(3), 268-282. doi:10.1037/1082-989X.12.3.268
What Works Clearinghouse. (2014). WWC procedures and standards handbook version 3.0. Retrieved from http://ies.ed.gov/ncee/wwc/Handbooks
Wolbers, K. A., Dostal, H. M., Graham, S., Cihak, D., Kilpatrick, J. R., & Saulsburry, R. (2015). The writing performance of elementary students receiving strategic and interactive writing instruction. Journal of Deaf Studies and Deaf Education, 20(4), 385-398. doi:10.1093/deafed/ env022
Wong, L.-H., & Looi, C.-K. (2011). What seams do we remove in mobile assisted seamless learning? A critical review of the literature. Computers & Education, 57(4), 2364-2381. doi:10.1016/j. compedu.2011.06.007
Wu, W.-H., Wu, Y.-C., Chen, C.-Y., Kao, H.-Y., Lin, C.-H., & Huang, S.-H. (2012). Review of trends from mobile learning studies: A meta-analysis. Computers & Education, 59(2), 817-827. doi: 10.1016/j.compedu.2012.03.016
Yen, J.-C., Lee, C.-Y., & Chen, I-J. (2012). The effects of image-based concept mapping on the learning outcomes and cognitive processes of mobile learners. British Journal of Educational Technology, 43(2), 307-320. doi:10.1111/j.1467-8535.2011.01189.x
Zurita, G., & Nussbaum, M. (2004). Computer supported collaborative learning using wirelessly interconnected handheld computers. Computers & Education, 42(3), 289-314. doi:10.1016/j. compedu.2003.08.005
Zurita, G., Nussbaum, M., & Salinas, R. (2005). Dynamic grouping in collaborative learning supported by wireless handhelds. Educational Technology & Society, 8(3), 149-161. Retrieved from http://www.ifets.info/journals/8_3/14.pdf

APA Format	Lee, H.-Y., Yang, J.-M., & Sung, Y.-T. (2017). Quality assessment and situational analysis of experimental e-learning designs: A case study of mobile learning. Journal of Research in Education Sciences, 62(2), 31-60. doi:10.6209/JORIES.2017.62(2).02

CONTRIBUTORSNOTICE

Journal directory listing - Volume 62 (2017) - Journal of Research in Education Sciences【62(2)】June (Special Issue: Digital Learning: Retrospect and Prospect) Back

CONTRIBUTORS
NOTICE

Journal directory listing - Volume 62 (2017) - Journal of Research in Education Sciences【62(2)】June (Special Issue: Digital Learning: Retrospect and Prospect)

Back