Journal of The Korean Association For Science Education (한국과학교육학회지)

The Korean Association for Science Education (한국과학교육학회)
권호 표지
DOI QR코드

DOI QR Code

Development and Validation of a Learning Progression for Astronomical Systems Using Ordered Multiple-Choice Items

순위 선다형 문항을 이용한 천문 시스템 학습 발달과정 개발 및 타당화 연구

  • Received : 2014.08.26
  • Accepted : 2014.12.26
  • Published : 2014.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

This study sought to investigate learning progressions for astronomical systems which synthesized the motion and structure of Earth, Earth-Moon system, solar system, and the universe. For this purpose we developed ordered multiple-choice items, applied them to elementary and middle school students, and provided validity evidence based on the consequence of assessment for interpretation of learning progressions. The study was conducted according to construct modeling approach. The results showed that the OMCs were appropriate for investigating learning progressions on astronomical systems, i.e., based on item fit analysis, students' responses to items were consistent with the measurement of Rasch model. Wright map analysis also represented that the assessment items were very effective in examining students' hypothetical pathways of development of understanding astronomical systems. At the lower anchor of the learning progression, while students perceived the change of location and direction of celestial bodies with only two-dimensional earth-based view, they failed to connect the locations of celestial bodies with Earth-Moon system model, and they could recognized simple patterns of planets in the solar system and milky way. At the intermediate levels, students interpreted celestial motion using the model of Earth rotation and revolution, Earth-Moon system, and solar system with space-based view, and they could also relate the elements of astronomical structures with the models. At the upper anchor, students showed the perspective change between space-based view and earth-based view, and applied it to celestial motion of astronomical systems, and they understood the correlation among sub-elements of astronomical systems and applied it to the system model.

이 연구에서는 지구, 지구 - 달 계, 태양계, 그리고 은하와 우주를 포함하는 천체의 구조와 운동을 종합한 천문 시스템(astronomical systems)의 학습 발달과정(learning progressions)을 조사하기 위하여, 순위 선다형 평가 문항을 개발하고 이를 초등/중학생들에게 적용한 검사 결과에 기반하여 학습 발달과정 해석의 타당성 검증의 근거를 제시하였다. 연구의 과정은 '구인특화, 평가문항 개발, 평가결과 기술, 측정 모델'로 이루어진 4 단계의 구인 모델링 방식(construct modeling approach)에 기반하여 진행되었다. 천문 시스템의 내용적 평가 구인으로 천체의 운동과 구조를 선정하였고, 탐구실행적 평가 구인으로 공간적 사고와 시스템 사고와 같은 천문학적 사고를 선정하였다. 이 연구에서 개발된 순위 선다형 평가 문항들은 천문 시스템에 대한 학습 발달과정을 조사하기 위한 검사 도구로서 적절하였다. 즉, item fit 분석 결과는 학생들의 문항 반응 결과가 Rasch 모델로 측정한 결과와 부합하였다. 그리고 Wright map 분석 결과는 이 연구의 평가 문항들이 천문시스템에 대한 학생들의 가설적 발달 경로를 조사하는데 매우 효과적임을 보여주었다. 학습 발달과정의 하위 정착점에서 학생들은 천체들의 위치와 운동 방향의 변화를 지구에서 보는 관점에서 2차원 평면으로 해석하는 공간적 사고 수준을 보였으며, 지구-달 계의 모델에서 천체의 위치와 변화를 부정확하게 연결하거나, 태양계 행성들과 은하수에 대한 단순한 패턴을 인식하는 시스템 사고 수준을 보였다. 학습발달과정의 중간 단계에서 학생들은 우주에서 내려다 본 지구 및 지구-달 계, 태양계의 모델을 근거로 천체의 운동을 해석하는 공간적 사고의 수준을 나타냈으며, 천체의 구조를 구성하는 요소들을 이 모델들과 연관시키는 시스템 사고를 보였다. 학습 발달과정의 상위 정착점에서 학생들은 우주에서 내려다 본 관점에서 형성된 천체의 운동 모델을 지구에서 바라 본 관점에서 얻은 자료와 서로 연관시키는 관점 전환의 공간적 사고를 지구의 운동, 지구-달 계의 운동, 태양계 행성의 운동, 은하와 우주의 운동에 적용할 수 있었다. 또한 상위 정착점에 도달한 학생들은 천문 시스템의 설명 모델을 구성하는 하위 요소들 간의 상호연관성을 파악하고 이를 설명 모델에 적용할 수 있는 시스템 사고를 나타냈다.

Keywords

References

  1. Achieve, Inc. (2013). Next Generation Science Standards. Achieve Inc. On behalf of the twenty-six states and partners that collaborated on the NGSS.
  2. AERA, APA, & NCME. (1999). Standards for educational and psychological testing. Washington, DC: AERA.
  3. Alonzo, A. C., & Steedle J. T. (2009). Developing and assessing a force and motion learning progression. Science Education, 93(3), 389-421. https://doi.org/10.1002/sce.20303
  4. Ben-Zvi Assaraf, O., & Orion, N. (2005). Development of system thinking skills in the context of Earth System education. Journal of Research in Science Teaching, 42, 518-560. https://doi.org/10.1002/tea.20061
  5. Ben-Zvi Assaraf, O., & Orion, N. (2010). System thinking skills at the elementary school level. Journal of Research in Science Teaching, 47, 540-563.
  6. Berland, L.K., & McNeill, K.L. (2010). A learning progression for scientific argumentation: Understanding student work and designing supportive instructional contexts. Science Education, 94, 765-793. https://doi.org/10.1002/sce.20402
  7. Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5, 7-74.
  8. Black, P., Wilson, M., & Yao, S. Y. (2011). Road maps for learning: A guide to the navigation of learning progressions. Measurement: Interdisciplinary Research & Perspective, 9, 71-123. https://doi.org/10.1080/15366367.2011.591654
  9. Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd edition). New York, NY: Routledge.
  10. Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. New York, NY: Springer.
  11. Briggs, D. C., & Alonzo, A. C. (2012). The psychometric modeling of ordered multiple-choice item responses for diagnostic assessment with a learning progression. In A. C. Alonzo & A. W. Gotwals (Eds.), Learning progressions in science: Current challenges and future directions (pp. 293-316). Rotterdam, The Netherlands: Sense Publishers.
  12. Briggs, D. C., Alonzo, A. C., Schwab, C., & Wilson, M. (2006). Diagnostic assessment with ordered multiple-choice items. Educational Assessment, 11(1), 33-63. https://doi.org/10.1207/s15326977ea1101_2
  13. Corcoran, T., & Silander, M. (2009). Instruction in high schools: The evidence and the challenge. The Future of Children: America's High Schools, 19, 157-183.
  14. Corcoran, T., Mosher, F. A., & Rogat, A. (2009). Learning progressions in science: An evidence based approach to reform. Consortium for Policy Research in Education Report #RR-63. Philadelphia, PA: Consortium for Policy Research in Education.
  15. Duncan, R. G., Rogat, A. D., & Yarden, A. (2009). A learning progression for deepening students' understandings of modern genetics across the 5th-10th grades. Journal of Research in Science Teaching, 46, 655-674. https://doi.org/10.1002/tea.20312
  16. Duschl, R. A., & Grandy, R. (Eds.) (2008). Teaching scientific inquiry: Recommendations for research and implementation. Rotterdam, The Netherlands: Sense Purblishers.
  17. Duschl, R., Maeng, S., & Sezen, A. (2011). Learning progressions and teaching sequences: A review and analysis. Studies in Science Education, 47, 123-182. https://doi.org/10.1080/03057267.2011.604476
  18. Furtak, E. M. (2012). Linking a learning progression for natural selection to teachers' enactment of formative assessment. Journal of Research in Science Teaching, 49(9), 1181-1210. https://doi.org/10.1002/tea.21054
  19. Gotwals, A. W., & Songer, N. B. (2013). Validity evidence for learning progression-based assessment items that fuse core disciplinary ideas and science practices. Journal of Research in Science Teaching, 50, 597-626. https://doi.org/10.1002/tea.21083
  20. Ji, E., & Chae, S. (2000). Theory and practices of Rasch model. Seoul: Kyoyook-Kwahak-Sa.
  21. Jin, H., & Anderson, C. W. (2012). A learning progression for energy in socio-ecological systems. Journal of Research in Science Teaching, 49, 1149-1180. https://doi.org/10.1002/tea.21051
  22. Maeng, S., Seong, Y., & Jang, S. (2013). Present states, methodological features, and an exemplar study of the research on learning progressions. Journal of the Korean Association for Science Education, 33, 161-180. https://doi.org/10.14697/jkase.2013.33.1.161
  23. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149-174. https://doi.org/10.1007/BF02296272
  24. National Research Council (NRC). (2006). Learning to think spatially. Washington, DC: National Academies Press.
  25. National Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: National Academies Press
  26. National Research Council. (2012). A framework for k-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: National Academies Press.
  27. Neumann, K., Viering, T., Boone, W. J., & Fischer, H. E. (2013). Towards a learning progression of energy. Journal of Research in Science Teaching, 50, 162-188. https://doi.org/10.1002/tea.21061
  28. Orion, N., & Basis, T. (2008). Characterization of High School Students' System Thinking Skills in the Context of Earth Systems. Presented in the 2008 NARST Annual Meeting. March, 2008. Baltimore, U.S.A.
  29. Plummer, J. D. (2014). Spatial thinking as the dimension of progress in an astronomy learning progression. Studies in Science Education, 50(1), 1-45.
  30. Plummer, J. D., & Krajcik, J. (2010). Building a learning progression for celestial motion: Elementary levels from an Earth-based perspective. Journal of Research in Science Teaching, 47(7), 768-787. https://doi.org/10.1002/tea.20355
  31. Schwarz, C. V., Reiser, B. J., Davis, E. A., Kenyon, L., Acher, A., Fortus, D., Shwartz, Y., Hug, B., & Krajcik, J. (2009). Developing a learning progression for scientific modeling: Making scientific modeling accessible and meaningful for learners. Journal of Research in Science Teaching, 46, 632-654. https://doi.org/10.1002/tea.20311
  32. Seong, T. (2002). Validity and Reliability. Seoul: Hakjisa.
  33. Seong, T., & Si, K. (2006). Reasearch Methodology. Seoul: Hakjisa.
  34. Shea, N. A., & Duncan, R. G. (2013). From theory to data: The process of refining learning progressions. Journal of the Learning Sciences, 22, 7-32. https://doi.org/10.1080/10508406.2012.691924
  35. Songer, N. B., & Gotwals, A. W. (2012). Guiding explanation construction by children at the entry points of learning progressions. Journal of Research in Science Teaching, 49, 141-165. https://doi.org/10.1002/tea.20454
  36. Stevens, S., Delgado, C., & Krajcik, J. S. (2010). Developing a hypothetical multi-dimensional learning progression for the nature of matter. Journal of Research in Science Teaching, 47, 687-715.
  37. Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, NJ: Lawrence Erlbaum Associates.
  38. Wilson, M. (2009). Measuring progressions: Assessment structures underlying a learning progression. Journal of Research in Science Teaching, 46, 716-730. https://doi.org/10.1002/tea.20318
  39. Wilson, M. (2012). Responding to a challenge that learning progressions pose to measurement practice. In A. C. Alonzo & A. W. Gotwals (Eds.), Learning progressions in science: Current challenges and future directions (pp. 317-343). Rotterdam, The Netherlands: Sense Publishers.
  40. Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit value. Rasch Measurement Transactions, 8(3), 370.

Cited by

  1. Cross-Sectional Item Response Analysis of Geocognition Assessment for the Development of Plate Tectonics Learning Progressions: Rasch Model vol.35, pp.1, 2015, https://doi.org/10.14697/jkase.2015.35.1.0037
  2. The Effects of an Instruction Using Geologic Planar Figures on High School Students’ Ability of Spatial Visualization and Geologic Spatial Ability vol.36, pp.3, 2015, https://doi.org/10.5467/JKESS.2015.36.3.280
  3. An Analysis of Systems Thinking Revealed in Middle School Astronomy Classes: The Case of Science Teachers’ Teaching Practices for the Unit of Stars and Universe vol.36, pp.6, 2015, https://doi.org/10.5467/JKESS.2015.36.6.591
  4. Effects of Systems Thinking on High School Students’ Science Self-Efficacy vol.37, pp.3, 2016, https://doi.org/10.5467/JKESS.2016.37.3.173
  5. A Proposal of Curriculum and Teaching Sequence for Seasonal Change by Exploring a Learning Progression vol.39, pp.3, 2018, https://doi.org/10.5467/JKESS.2018.39.3.260
  6. Exploring 6th Graders Learning Progression for Lunar Phase Change: Focusing on Astronomical Systems Thinking vol.39, pp.1, 2018, https://doi.org/10.5467/JKESS.2018.39.1.103
  7. 생태계에 대한 학습발달과정의 개발과 평가 vol.36, pp.1, 2016, https://doi.org/10.14697/jkase.2016.36.1.0029
  8. 초등학생의 지구의 운동과 태양계 학습 발달과정의 타당성 검증: 구인 타당도 및 결과 타당도를 중심으로 vol.36, pp.1, 2014, https://doi.org/10.14697/jkase.2016.36.1.0177
  9. 용해와 용액 개념에 대한 학습발달과정 조사 vol.36, pp.2, 2014, https://doi.org/10.14697/jkase.2016.36.2.0295
  10. 기초공통개념으로서 에너지에 대한 3~9학년 학생들의 문항 반응 분석 vol.36, pp.6, 2016, https://doi.org/10.14697/jkase.2016.36.6.0815
  11. 지구과학 탐구에서 통합 탐구 기능에 대한 학습발달과정 탐색 vol.38, pp.3, 2014, https://doi.org/10.5467/jkess.2017.38.3.222
  12. 천체투영관 수업이 학생들의 천문 개념 이해에 미치는 효과 vol.42, pp.1, 2014, https://doi.org/10.21796/jse.2018.42.1.49
  13. 학습발달과정에 근거한 과정중심 STEAM 역량 평가 모델에 대한 이론적 탐색 vol.42, pp.2, 2018, https://doi.org/10.21796/jse.2018.42.2.132
  14. 지구의 공전과 별자리의 겉보기 운동에 대한 초등학생들의 공간적 추론 발달 경로의 사례 연구 vol.38, pp.4, 2014, https://doi.org/10.14697/jkase.2018.38.4.481
  15. 천문학적 사고를 반영한 천문교육 프로그램의 개발 및 적용: 과학관 천체 투영관 수업 사례 vol.40, pp.1, 2019, https://doi.org/10.5467/jkess.2018.40.1.86
  16. 문항 반응 분석을 활용한 초등학생과 중학생의 시스템 사고 검사 도구 타당도 검증 vol.67, pp.2, 2014, https://doi.org/10.25152/ser.2019.67.2.249
  17. 고등학생들의 통합 탐구 기능 향상을 위한 인지적 스캐폴딩 도구 개발 및 적용 vol.39, pp.4, 2014, https://doi.org/10.14697/jkase.2019.39.4.545
  18. Feedback Loop Reasoning and Knowledge Sources for Elementary Students in Three Countries vol.16, pp.2, 2020, https://doi.org/10.29333/ejmste/112582
  19. 다층 서답형 문항을 이용한 태양계 구조 학습 발달과정 개발 및 타당성 검증 vol.41, pp.3, 2020, https://doi.org/10.5467/jkess.2020.41.3.291
  20. 행성의 공전 운동에 대한 초등 예비교사의 이해와 설명 모델 vol.40, pp.1, 2014, https://doi.org/10.15267/keses.2021.40.1.1
  21. 국내외 천문 교육 프로그램 관련 연구 동향 비교 분석 vol.36, pp.1, 2021, https://doi.org/10.5303/pkas.2021.36.1.025