This study examined the quality of a teacher-made achievement test using Rasch analyses and provided useful assessment feedback information for improving teaching and testing. The participants were 8,936 seventh-grade students (4,617 boys and 4,319 girls) from the Kaohsiung area in Taiwan. Data were collected through a teacher-made biology achievement test and analyzed using the Rasch model. The results indicated that the test clearly discriminated students’ abilities. Some questions, however, were found to have unfair content for girls. Additionally, some mismatches were identified between teachers’ expectations and students’ assessments of question difficulty. This paper also provides assessment feedback diagrams, including a person–item map and diagnostic map, and a method of identifying “the zone of fundamental learning development” and “the zone of proximal learning development” is proposed to enable teachers to plan students’ learning progression.

» More

1. 王子華、范雅晴、王國華（2008）。數理科在職教師評量素養培育模式之探究。科學教育學刊，16（1），25-51。doi:10.6173/CJSE.2008.1601.04【Wang, T.-H., Fan, Y.-C., & Wang, K.-H. (2008). Implementation of web-based assessment system in improving in-service teachers’ assessment literacy. Chinese Journal of Science Education, 16(1), 25-51 doi:10.6173/ CJSE.2008.1601.04】
2. 教育部（2008）。教育部補助辦理精進教學要點修正規定。行政院公報，014（215）。取自https://gazette.nat.gov.tw/EG_FileManager/eguploadpub/eg014215/ch05/type2/gov40/num12/Eg.pdf【Ministry of Education. (2008). Revised regulations governing Ministry of Education subsidies for instruction improvement. The Executive Yuan Gazatte Online, 014(215). Retrieved from https://gazette.nat.gov.tw/EG_ FileManager/eguploadpub/eg014215/ch05/type2/gov40/num12/Eg.pdf】
3. Alonzo, A., & Steedle, J. T. (2009). Developing and assessing a force and motion learning progression. Science Education, 93(3), 389-421. doi:10.1002/sce.20303
4. Allal, L., & Ducrey, G. P. (2000). Assessment of-or in-the zone of proximal development. Learning and Instruction, 10(2), 137-152. doi:10.1016/S0959-4752(99)00025-0
5. Bell, B. (2007). Classroom assessment of science learning. In S. K. Abell & N. G. Lederman (Eds.), Handbook of research on science education (pp. 965-1006). Mahwah, NJ: LEA.
6. Bell, B., & Cowie, B. (2001). The characteristics of formative assessment in science education. Science Education, 85(5), 536-553. doi:10.1002/sce.1022
7. Bertenthal, M. W., & Wilson, M. R. (2006). Systems for state science assessment. Washington, DC: National Academy Press.
8. Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Working inside the black box: Assessment for learning in the classroom. Phi Delta Kappan, 86(1), 8-21. doi:10.1177/0031  72170408600105
9. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-74. doi:10.1080/0969595980050102
10. Black, P., & Wiliam, D. (2010). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 92(1), 81-90. doi:10.1177/003172171009200119
11. Boone, W. J., & Scantlebury, K. (2006). The role of Rasch analysis when conducting science education research utilizing multiple-choice tests. Science Education, 90(2), 253-269. doi:10. 1002/sce.20106
12. Boone, W. J., Townsend, J. S., & Staver, J. (2011). Using Rasch theory to guide the practice of survey development and survey data analysis in science education and to inform science reform efforts: An exemplar utilizing STEBI self-efficacy data. Science Education, 95(2), 258-280. doi:10.1002/sce.20413
13. Briggs, D. C., Alonzo, A. C., Schwab, C., & Wilson, M. (2006). Diagnostic assessment with ordered multiple-choice items. Educational Assessment, 11(1), 33-63. doi:10.1207/s15326977ea1101_2
14. Budoff, M. (1987). The validity of learning potential assessment. In C. S. Lidz (Ed.), Dynamic assessment: An interactional approach to evaluating learning potential (pp. 53-81). New York, NY: Guilford Press.
15. Claesgens, J., Scalise, K., Wilson, M., & Stacy, A. (2009). Mapping student understanding in chemistry: The perspectives on chemists. Science Education, 93(1), 56-85. doi:10.1002/sce. 20292
16. Duncan, R. G., & Gotwals, A. W. (2015). A tale of two progressions: On the benefits of careful comparisons. Science Education, 99(3), 410-416. doi:10.1002/sce.21167
17. Duncan, R. G., & Hmelo-Silver, C. E. (2009). Learning progressions: Aligning curriculum, instruction, and assessment. Journal for Research in Science Teaching, 46(6), 606-609. doi:10. 1002/tea.20316
18. Duncan, R. G., & Reiser, B. J. (2007). Reasoning across ontologically distinct levels: Students’ understandings of molecular genetics. Journal of Research in Science Teaching, 44(7), 938-959. doi:10.1002/tea.20186
19. Fulmer, G. W. (2014). Validating proposed learning progressions on force and motion using the force concept inventory: Findings from Singapore secondary schools. International Journal of Science and Mathematics Education, 13(6), 1235-1254. doi:10.1007/s10763-014-9553-x
20. Furtak, E. M., Morrison, D., & Kroog, H. (2014). Investigating the link between learning progressions and classroom assessment. Science Education, 98(4), 640-673. doi:10.1002/sce.21122
21. Gioka, O. (2009). Teacher or examiner? The tensions between formative and summative assessment in the case of science coursework. Research in Science Education, 39(4), 411-428. doi:10.1007/ s11165-008-9086-9
22. Graham, P. (2005). Classroom-based assessment: Changing knowledge and practice through preservice teacher education. Teaching and Teacher Education, 21(6), 607-621. doi:10.1016/ j.tate.2005.05.001
23. Griffin, P. (2007). The comfort of competence and the uncertainty of assessment. Studies in Educational Evaluation, 33(1), 87-99. doi:10.1016/j.stueduc.2007.01.007
24. Guthke, J. (1993). Developments in learning potential assessment. In J. H. M. Hamers & K. Sijtsma (Eds.), Learning potential assessment: Theoretical, methodological and practical issues (pp. 43-67). Amsterdam, The Netherland: Swets & Zeitlinger.
25. Hammer, D., & Sikorski, T. R. (2015). Implications of complexity for research on learning progressions. Science Education, 99(3), 424-431. doi:10.1002/sce.21165
26. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. doi:10.3102/003465430298487
27. Holland, P. W., & Wainer, H. (Eds.). (1993). Differential item functioning. Hillsdale, NJ: Lawrence Erlbaum Associates.
28. Kennedy, C. A., Wilson, M., & Draney, K. (2008). ConstructMap [Computer Program]. UC Berkeley, CA: BEAR Center.
29. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254-284. doi:10.1037/0033-2909.119.2.254
30. Lee, H.-S., & Liu, O. L. (2010). Assessing learning progression of energy concepts across middle school graded: The knowledge integration perspective. Science Education, 94(4), 665-688. doi:10.1002/sce.20382
31. Linacre, J. M., & Wright, B. D. (1994). Chi-square fit statistics. Rasch Measurement Transactions, 8(2), 350.
32. Lipnevich, A. A., & Smith, J. K. (2009). Effects of differential feedback on students’ examination performance. Journal of Experimental Psychology: Applied, 15(4), 319-333. doi:10.1037/ a0017841
33. Maclellan, E. (2004). Initial knowledge states about assessment: Novice teachers’ conceptualisations. Teaching and Teacher Education, 20(5), 523-535. doi:10.1016/j.tate.2004.04.008
34. Mertler, C. A. (2004). Secondary teachers’ assessment literacy: Does classroom experience make a difference? American Secondary Education, 33(1), 49-64.
35. Mertler, C. A. (2009). Teachers’ assessment knowledge and their perceptions of the impact of classroom assessment professional development. Improving Schools, 12(2), 101-113. doi:10. 1177/1365480209105575
36. Mohan, L., Chen, J., & Anderson, C. W. (2009). Developing a multi-year learning progression for carbon cycling in socio-ecological systems. Journal of Research in Science Teaching, 46(6), 675-698. doi:10.1002/tea.20314
37. Neumann, I., Fulmer, G. W., & Liang, L. L. (2013). Analyzing the FCI based on a force and motion learning progression. Science Education Review Letters, 2013, 8-14. doi:10.18452/8208
38. Ogan-Bekiroglu, F. (2009). Assessing assessment: Examination of pre-service physics teachers’ attitudes towards assessment and factors affecting their attitudes. International Journal of Science Education, 31(1), 1-39. doi:10.1080/09500690701630448
39. Otero, V. K., & Nathan, M. J. (2008). Preservice elementary teachers’ views of their students’ prior knowledge of science. Journal of Research in Science Teaching, 45(4), 497-523. doi:10.1002/ tea.20229
40. Plake, B. S., & Impara, J. C. (1996). Teacher assessment literacy: What do teachers know about assessment? In G. Phye (Ed.), Handbook of classroom assessment (pp. 53-68). San Diego, CA: Academic Press. doi:10.1016/B978-012554155-8/50005-3
41. Plummer, J. D., Palma, C., Flarend, A., Rubin, K., Ong, Y. S., Botzer, B., …Furman, T. (2015). Development of a learning progression for the formation of the solar system. International Journal of Science Education, 37(9), 1381-1401. doi:10.1080/09500693.2015.1036386
42. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Danmark: Institute of Educational Research.
43. Sato, M., Wei, R. C., & Darling-Hammond, L. (2008). Improving teachers’ assessment practices through professional development: The case of National Board Certification. American Educational Research Journal, 45(3), 669-700. doi:10.3102/0002831208316955
44. Shepard, L. A. (2005). Linking formative assessment to scaffolding. Educational Leadership, 63(3), 66-70.
45. Shouse, A. W., Schweingruber, H. A., & Duschl, R. A. (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: National Academy Press.
46. Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153-189. doi:10.3102/0034654307313795
47. Songer, N. B., Kelcey, B., & Gotwals, A. W. (2009). How and when does complex reasoning occur? Empirically driven development of a learning progression focused on complex reasoning about biodiversity. Journal of Research in Science Teaching, 46(6), 610-631. doi:10.1002/tea.20313
48. Steedle, J. T., & Shavelson, R. J. (2009). Supporting valid interpretations of learning progression level diagnoses. Journal of Research in Science Teaching, 46(6), 699-715. doi:10.1002/tea.20308
49. Taylor, C. S., & Nolen, S. B. (2008). Classroom assessment: Supporting teaching and learning in real classroom (2nd ed.). Upper Saddle River, NJ: Pearson Education.
50. Testa, I., Galano, S., Leccia, S., & Puddu, E. (2015). Development and validation of a learning progression for change of seasons, solar and lunar eclipses, and moon phases. Physical Review Special Topics-Physics Education Research, 11(2), 020102-1-020102-22. doi:10.1103/PhysRev STPER.11.020102
51. Treagust, D. F., Jacobwitz, R., Gallagher, J. L., & Parker, J. (2001). Using assessment as a guide in teaching for understanding: A case study of a middle school science class learning about sound. Science Education, 85(2), 137-157.
52. Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes (M. Cole, V. John-Steiner, S. Scribner, & E. Souberman, Eds. & Trans.). Cambridge, MA: Harvard University Press. (Original works published 1930-1935).
53. Wang, J.-R., Kao, H.-L., & Lin, S.-W. (2010). Preservice teachers’ initial conceptions about assessment of science learning: The coherence with their views of learning science. Teaching and Teacher Education, 26(3), 522-529. doi:10.1016/j.tate.2009.06.014
54. Wang, T.-H., Wang, K.-H., & Huang. S.-C. (2008). Designing a web-based assessment environment for improving pre-service teacher assessment literacy. Computers & Education, 51(1), 448-462. doi:10.1016/j.compedu.2007.06.010
55. Wang, T.-H., Wang, K.-H., Wang, W.-L., Huang, S.-C., & Chen, S.-Y. (2004). Web-based assessment and test analyses (WATA) system: Development and evaluation. Journal of Computer Assisted Learning, 20(1), 59-71. doi:10.1111/j.1365-2729.2004.00066.x
56. Wang, W.-C. (2008). Assessment of differential item functioning. Journal of Applied Measurement, 9(4), 387-408.
57. Warbruck, F., & Stachelscheid, K. (2010). Adaptive learning environment for chemical reactions. In G. Cakmakci & M. F. Tasar (Eds.), Contemporary science education research: Learning and assessment (pp. 425-429). Ankara, Turkey: Pegem Akademi.
58. Wilson, M. (2008). Cognitive diagnosis using item response models. Journal of Psychology, 216(2), 74-88. doi:10.1027/0044-3409.216.2.74
59. Wilson, M., & Carstensen, C. (2005). Assessment to improve learning in mathematics: The BEAR assessment system. Journal of Educational Research and Development, 1(3), 27-50.
60. Wilson, M., & Scalise, K. (2006). Assessment to improve learning in higher education: The BEAR assessment system. Higher Education, 52(4), 635-663. doi:10.1007/s10734-004-7263-y
61. Wilson, M., Scalise, K., Reitze, A. M. A., Bein, E., Bousselot, T., Gochyyev, P., …Vorp, R. (2010, May). Progress monitoring for real classroom contexts: The formative assessment delivery system. Paper presented at the annual meeting of the American Educational Research Association, Denver, CO.
62. Wu, M. L., Adams, R. J., & Wilson, M. R. (2007). ConQuest [Computer software and manual]. Camberwell, Australia: Australian Council for Educational Research.

APA Format	Chen, Y.-T., Ho, H.-C., Liu, K.-H., Lin, H.-S., & Cheng, Y.-Y. (2017). Glimpse into teaching and learning using Rasch analyses of a teacher-made science achievement test. Journal of Research in Education Sciences, 62(3), 1-23. doi:10.6209/JORIES.2017.62(3).01