evaluating the alignment of the intended and enacted school

Asia Pacific Journal on Curriculum Studies Vol. 3, Issue 2, December 2020

E ISSN 2672-3247 URL: https://apjcs.org

Evaluating the Alignment of the Intended and Enacted School-based Social Studies Curricula Using Seitz’s Methodological Framework

Emie Joane M. Guintu1 and Amelia C. Fajardo2

1Intermediate School Department, MGC New Life Christian Academy 2College of Education, University of the Philippines, Diliman, Quezon City, Philippines

How to cite Guintu, E. J., & Fajardo, A. C. (2020). Evaluating the Alignment of the Intended and Enacted School-based Social Studies Curricula Using Seitz’s Methodological Framework. Asia Pacific Journal on Curriculum Studies, 3(2), 11-20. https://doi.org/10.53420/apjcs.2020.5 Abstract. This study determined whether Seitz’s Methodological Framework could be used to evaluate the alignment between the intended and enacted school-based curricula of Araling Panlipunan 3 or Social Studies 3 in the Philippine setting. In the intended curriculum, the cognitive processes called for by the learning competencies were determined using the Delphi Method while for the enacted curriculum, data was gathered using classroom observations, teacher survey, lesson plans, and teacher interview. The results of the study showed that for the content dimension, there was 100% alignment between the intended and the enacted curricula, and for the cognitive process dimension, there was only 57.89% alignment between the intended and the enacted curricula. Through this study, the usefulness of Seitz’s Methodological Framework was examined using the following parameters: process, achievement of goal, and ease of use or practicality. Keywords: curriculum alignment, intended curriculum, enacted curriculum Introduction

According to Porter (2002), a curriculum is divided into three parts: the intended curriculum which society decides as important to learn; the enacted curriculum or that which is implemented or taught in the classroom; and, the assessed curriculum which is what the students actually learned. This is in congruence with the three components of the curriculum proposed by English (1992), namely: the written, the taught, and the tested curricula which are being examined in alignment studies. The alignment or the extent to which the intended, the enacted, and the assessed curricula agree and work together is a crucial element in the achievement of curricular goals (Ananda, 2003a; Resnick, Rothman, Slattery, & Vranek, 2003; Webb, 1997b). In the Philippine setting, misalignment can occur between the intended and the enacted curricula because of the difficulties encountered by the teachers as they unpack the national curriculum provided by the Department of Education (DepEd) (Urbiztondo, 2017). The teacher’s misinterpretation of the intended curriculum can impact its implementation inside the classroom. However, this problem can be addressed by alignment studies since results of an alignment study might be helpful in identifying areas in the three components of the curriculum that need to be clarified, restructured, or changed to achieve alignment (Tindal, Cipoletti, & Almond, 2005). Studies on curriculum alignment between the intended and the assessed curricula (Moss, 1999; Webb, 2002; McGehee & Griffith, 2001), and/or between the enacted and the assessed curricula (Blank, Porter, & Smithson, 2001; Keohane, 2008) have already been done. Yet, only a few alignment studies have been conducted between the intended and the enacted curricula because majority of recently developed alignment models focus more on analyzing the alignment between standards (intended) and assessments (assessed) (Näsström, 2008). In a study done by Dr. Paolina Seitz (2014, 2017), curriculum alignment among the intended, the enacted, and the assessed curricula in Grade 9 Mathematics in Southern Alberta, Canada was examined, focusing on the domains of content/operations and cognitive processes. She used the Delphi Method to establish the cognitive levels for the intended curriculum. On the other hand, she administered classroom observations, conducted teacher interviews and teacher surveys based on the Survey of the Enacted Curriculum (SEC) Model (Porter & Smithson, 2001), and reviewed teacher unit plans and assessments administered during the instructional period in order to accurately record the enacted curriculum. To examine the assessed curriculum, teacher’s end - of - the - unit test, teacher survey, and assessments administered during the instructional period were used. The


12

methodological framework used by Seitz provided a model for future researches on alignment among the intended curriculum, the enacted curriculum, and the assessed curriculum (Seitz, 2014, 2017). Since Seitz was able to evaluate the alignment of the intended, the enacted, and the assessed curricula, the researcher would like to find if the methodological framework she used could be employed in evaluating the alignment between the first two curriculum components only, which are the intended and the enacted curricula. For purposes of this study, the enacted curriculum is referred to as such and not as the implemented or taught curriculum, although they are actually one and the same. Therefore, this study aimed to contribute to the growing literature on Curriculum Studies by determining whether the methodological framework used by Seitz (2014, 2017) to evaluate curriculum alignment in Mathematics 9 in Southern Alberta, Canada may also be used in evaluating the alignment of the intended and the enacted school-based curricula of Araling Panlipunan 3 or Social Studies 3. Data gathering particularly for the information about the enacted curriculum, which involves classroom observations, was conducted in the fourth quarter of school year 2018 to 2019. Moreover, the research also aimed to examine the usefulness of Seitz’s Methodological Framework in investigating curriculum alignment as well as to find ways by which it can be improved. Specifically, this study answered the following questions: 1. What is the degree of alignment of the intended and enacted curricula of the content and cognitive processes of Araling

Panlipunan 3? 2. What levels of cognitive processes correspond to the learning competencies of Araling Panlipunan 3? 3. How useful is the model used by Seitz in evaluating the alignment of the intended and enacted school-based curricula? 4. In what ways can Seitz’s Methodological Framework be improved? Method This study used Seitz’s (2014, 2017) Methodological Framework which employs a mixed-method design, particularly the convergent design of Cresswell & Plano Clark (2011). This involved: 1) simultaneous collection of data for the intended and the enacted curricula, 2) data analyses, 3) merging of results, and, 4) interpretation of merged results. Intended curriculum data collection The Delphi Method (Dalkey & Helmer, 1963) was used in this study to determine the cognitive processes (cognitive dimension) called for by the 19 learning competencies (content dimension) that were taught in the school where this study was conducted during the 4th quarter of Araling Panlipunan 3. The learning competencies came from DepEd’s Araling Panlipunan 3 Curriculum Guide. Eight expert/judges matched the learning competencies to their corresponding cognitive processes using the taxonomy table adopted from the “Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives” (Anderson, Krathwohl, Airasian, Cruikshank, Mayer, Pintrich, Raths, & Wittrock, 2001). The process was completed in three rounds. During the first round, each judge received the taxonomy table matrix and the Manual for the Delphi Method. Afterwards, the judges returned the accomplished taxonomy table matrix to the researcher who then computed for the proportion of judges who placed the learning competency in the same cell in the taxonomy table. The anonymous responses were combined and then sent back to the judges together with their initial response. Each judge was given an opportunity to see how he/she fared relative to the other judges. For the second and third round, each judge was provided a chance to adjust his/her previous judgment before returning it once again to the researcher. Upon reaching 80% to 100% consensus among the placements or judgments, the procedure was stopped (Seitz, 2014, 2017). Enacted curriculum data collection For this study, convenience sampling was used for classroom observations to be feasible for the researcher. The following instruments were used to gather data about the enacted curriculum: teacher survey, classroom observations recorded through a checklist, teacher interview, and the taxonomy table. There were 4 sections in Grade 3, with one teacher handling Araling Panlipunan 3 in all sections. This teacher was observed in all classes each day she taught during the fourth quarter. The teacher also provided lesson plans to the researcher. A checklist was used during the observations which allowed the observers to check off the learning competencies that were observed and the cognitive processes taught by the teacher during instruction. Daily observations for each class were done for the entire duration of the fourth quarter. The initial visit was a warm-up observation for the students to get used to the idea of having a visitor inside the classroom. The rest of the visits were used for classroom observations (Seitz, 2014, 2017). The main observer in this study was the


13

researcher assisted by 3 on-call classroom observers who were Araling Panlipunan teachers in grade school. The observations were conducted within 10 weeks from February 18 to April 26, 2019. Each section had four 40-minute sessions per week. A total of 140 observations were done for the entire fourth quarter. After the observation period, the teacher answered a teacher survey. The teacher survey was divided into 3 parts. In Part I, the teacher indicated the following for each learning competency: whether she has provided the students an opportunity to learn the specified learning competency, and the highest possible cognitive process that is called for by the learning competency. In Part II, the teacher answered questions pertaining to her planning and teaching practices. For Part III, the teacher answered questions about teacher characteristics. A teacher interview was also conducted after the the survey to discuss some information that needed to be clarified. Results and Discussion The Porter Alignment Index was used to compute for the degree of alignment between the intended and the enacted curricula. The Araling Panlipunan Content Index is the ratio of the total number of sections where the teacher taught the learning competency to the total number of learning competencies in the unit examined. Moreover, the Araling Panlipunan Cognitive Process Index is the ratio of the total number of sections where the teacher taught the intended cognitive process for the learning competency to the total number of learning competencies in the unit examined. An index close to or equal to 1.00 indicates full curriculum alignment for both the content and cognitive process indices. The intended curriculum For the fourth quarter, there were 19 learning competencies which were supposed to be covered for Araling Panlipunan 3 in the school where the study was conducted. These learning competencies which were taken from the K-12 Curriculum Guide provided by the Department of Education made up the intended curriculum. But since the judges commented that four of the learning competencies belong to the affective dimension and that they could not match them to a cognitive process indicated in the taxonomy table, these learning competencies were transformed so that matching each of them with a cognitive process could be possible. The intended curriculum was made up of two dimensions – the content dimension and the cognitive dimension. The content dimension was stated as learning competencies while the cognitive dimension was the cognitive processes (remember, understand, apply, analyze, evaluate, or create) a student was expected to engage in while learning the competency required. However, since the cognitive processes were not explicitly stated in the K-12 Curriculum Guide from DepEd, they were determined by judges or a panel of experts who reached a consensus through the use of the Delphi Method. The Delphi Process was completed in three rounds and took three months and 24 days from March 18 to July 9, 2019. On the third round, seven learning competencies reached 100% consensus while ten reached 87.5% consensus. However, two learning competencies reached only 75% consensus . Hence, out of the 19 learning competencies, only 17 reached consensus that was above 80%. The Delphi Process was stopped after the third round since some of the judges had already expressed that they would not be changing their answers anymore. The enacted curriculum The following results came from the data gathered about the enacted curriculum. Classroom Observation (Content and Cognitive Process). For each of the 4 sections in Grade 3, the teacher taught a total of 16 learning competencies out of the 19 learning competencies she was supposed to teach for the fourth quarter. It was noted that the cognitive processes taught for each learning competency were the same across all four sections in Grade 3. Nevertheless, the length of time spent on teaching the cognitive processes was not exactly the same since it depended on how fast the students in each section were able to master what was being taught. Teacher Survey and Teacher Interview. The teacher indicated that she has provided the students an opportunity to learn 16 out of 19 learning competencies for the fourth quarter, which was the same with the number of learning competencies the observers saw being taught in the class. During the interview, the teacher’s curriculum unpacking practices came to light. She mentioned that she used the Araling Panlipunan 3 Curriculum Guide from DepEd when she planned what she was going to do in the fourth quarter. First, she looked


14

into the verbs used in the learning competencies to know the skills her students should be able to acquire. Second, she included other learning competencies aside from those indicated in the curriculum guide with the permission of her subject coordinator. Third, she went through all the learning competencies and combined those that could be merged in order to maximize time. Fourth, she looked at the available references. Fifth, she proceeded to come up with her motivation activity before focusing on the discussion part of the lesson plan, putting into consideration the students’ background. Sixth, she planned how to go about with the discussion, making sure to use the simplest words to teach in the simplest way possible by giving examples. She used several activities in the fourth quarter such as: historical re-enactment, speech choir, and film/video showing. Lastly, she chose the assessment she would use in order to measure student learning. However, the teacher mentioned that due to time constraint, she did not really focus on ensuring curriculum alignment, assuming that alignment should already exist because the learning competencies she taught in class came from the curriculum guide, and she made sure to cover as much of them as possible. When asked what she knows about curriculum alignment, the teacher mentioned that in one of her master’s (MA) classes, she learned that curriculum is a broad concept which involves curriculum design, content/topic, what students should possess at the end of each grade level, and teaching strategies. She said that curriculum alignment is a higher level of curriculum study and that when you say “aligned”, you are making sure that what you teach is what you are really supposed to teach. When you have evaluated alignment and you found out it is not aligned, she said that the teacher has to make some adjustments in order to reach the level of what she is expected to teach. However, she maintained that she could not remember attending any class, training, or workshop about curriculum alignment in the past. Findings At the end of the study, the results revealed that there was 100% curriculum alignment between the intended and the enacted curricula of Araling Panlipunan 3 for the content dimension and a 57.89% curriculum alignment for the cognitive process dimension. Moreover, fifteen out of the nineteen learning competencies included in this study required cognitive processes that belong to low - level thinking skills (remember and understand) while only four learning competencies required cognitive processes that belong to higher - order thinking skills (apply, analyze, and create). Also, it was found that Seitz’s Methodological Framework is useful in evaluating the alignment of the intended and enacted curricula in terms of process and achievement of goals. On the other hand, in terms of ease of use or practicality, it was deemed time-consuming due to the length of time spent on classroom observations and the Delphi Procedure. It was also costly in terms of the manpower needed if the observation is to be done for one school year. In addition , the teacher’s knowledge of what each cognitive process actually means has an impact on the validity of his/her answers in the teacher survey. The Framework may be improved by doing the following: allow a lower number of experts (less than 10) to serve as judges; include additional verbs in the taxonomy table for judges and observers to easily identify the cognitive processes that best fit each learning competency; have classroom observers whose sole purpose in the school is to measure curriculum alignment; and, orient teacher/s observed about what each cognitive process means prior to the accomplishment of the teacher survey. Lastly, it can be said that the teacher observed was indeed following the learning competencies prescribed by DepEd, however, the cognitive processes called for by the learning competencies were not all taught to the students. Conclusions and Recommendations After using Seitz’s Methodological Framework in evaluating the alignment between the intended and enacted school-based curricula of Araling Panlipunan 3, it can be concluded that curriculum alignment between the intended and the enacted school-based curricula of Araling Panlipunan 3 in terms of the content dimension was 100%; but, only 57.89% curriculum alignment was found in terms of the cognitive process dimension. These show that the teacher indeed taught the learning competencies of the intended curriculum but failed to teach all the cognitive processes called for by these learning competencies. Next, out of 19 learning competencies, fifteen called for low - level thinking skills while only four called for higher- order thinking skills. Also, Seitz’s Methodological Framework is useful in evaluating the alignment between the intended and the enacted school-based curricula of Araling Panlipunan 3 in the Philippine setting in terms of process and achievement of goals. The use of Cresswell & Plano Clark’s (2011) convergent design which is a mixed method design showed that the framework used by Seitz has a clear process of data collection, analysis, merging of results, and interpretation of merged results. Similarly, the Delphi Method has shown a clear process of how consensus can be achieved among judges through several rounds in the collection of data for the intended curriculum. Also, it was able to compute for the content and cognitive process alignment indices because the taxonomy table and the Delphi Method allowed for reaching consensus in terms of which cognitive processes were called for by the learning competencies in the intended curriculum while classroom observations permitted the observers to really see what was being taught by the teacher. On the other hand, in terms of ease of use or practicality, the framework was deemed challenging to use because of the following reasons: the length of time consumed in classroom observations as well as in the Delphi Procedure; difficulty in reaching 80% to 100% consensus in all of the learning competencies as shown in the results of this study wherein only 17 out of


15

the 19 learning competencies reached 87.5% to 100% consensus while two learning competencies reached only 75% consensus; added cost of hiring a teacher who will conduct classroom observation; and, dependence on teacher knowledge of cognitive processes for validity of the survey. Lastly, there are certain areas of the framework that may still be improved through: allowing a lower number of experts (less than 10) to serve as judges; including additional verbs in the taxonomy table for judges and observers to easily identify the cognitive processes that best fit each learning competency; having classroom observers whose sole purpose in the school is to measure curriculum alignment; and, orienting teacher/s observed about the cognitive processes in the teacher survey. Based on the results of the study, the following recommendations are deemed important by the researchers not only to improve Seitz’s Methodological Framework but also to enhance curriculum alignment in both private and public schools in order to positively impact teacher training or education and student learning. First, all teachers must undergo trainings on curriculum alignment to help them understand the need for alignment and to have enough knowledge on how to ensure that the intended, enacted, and assessed curricula are indeed in congruence with one another. Second, to better evaluate curriculum alignment in schools, the school officials may consider: having a department that solely focuses on monitoring horizontal and vertical alignment of the entire school-based curriculum to improve student learning (Gamoran, Porter, Smithson, & White, 1997); and/or empowering subject coordinators through curriculum alignment trainings or workshops. Third, the DepEd has to clearly establish or indicate the cognitive processes called for by the learning competencies. Fourth, the learning competencies in the curriculum guide must be clearly stated by using verbs that are specific and measurable. Fifth, DepEd may work on improving the learning competencies by ensuring that these target higher - order thinking skills instead of simply focusing on remembering and understanding. And lastly, the results of this study may be used as a basis for other curriculum alignment studies in the future. However, the results of this study cannot be generalized for other school-based curricula or other subject areas since the context and locale surrounding this research are specific to the school where the study was conducted.

References Achieve, Inc. (2001). Measuring Up: A report on education standards and assessments for Massachusetts. Retrieved from

http://www.achieve.org/files/MassachusettsBenchmarking10-2001.pdf American Educational Research Association (AERA), American Psychological Association (APA), & National Council on

Measurement in Education (NCME). (1999). Standards for educational and psychological testing. Washington, DC: Author.

Ananda, S. (2003a). Achieving alignment. Leadership, 33(1), 18–22. Ananda, S. (2003b). Rethinking issues of alignment under No Child Left Behind. San Francisco: WestEd. Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., Raths, J., & Wittrock, M. C.

(2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s Taxonomy of Educational Objectives. Boston, MA: Allyn & Bacon.

Anderson, L. W. (2002). Curricular alignment: A re-examination. Theory into Practice, 41(4), 255–260. Assessed Curriculum. (2016, March 1). Retrieved from http://www.ibe.unesco.org/en/glossary-curriculum-terminology/i/assessed-

curriculum. Australian Curriculum, Assessment and Reporting Authority (ACARA). (2009). Shape of the Australian Curriculum:

Mathematics. Retrieved May 31, 2018 from http://www.acara.edu.au/_resources/Australian_Curriculum_-_Maths.pdf Bhola, D. S., Impara, J. C., & Buckendahl, C. W. (2003). Aligning Tests with content Standards: Methods and Issues. Educational

Measurement, Issues and Practice, 2003(22), 21-29. Biggs, J.B. (1996). Enhancing teaching through constructive alignment. Higher Education, 32, 1-18. Biggs, J. B. (1999). Teaching for quality learning at university. Bucking- ham, England: Society for Research in Higher Education

and Open University Press.


16

Biggs, J., & Tang, C. (2007). Using Constructive Alignment in Outcomes-Based Teaching and Learning Teaching for Quality Learning at University (3rd ed., pp. 50-63). Maidenhead: Open University Press.

Blank, R. K., Porter, A. C., & Smithson, J. L. (2001). New tools for analyzing teaching, curriculum and standards in mathematics

and science. Washington, DC: Council of Chief State School Officers. Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H. & Krathwohl, D. R. (1956). Taxonomy of educational objectives:

Handbook I: Cognitive domain. New York: David McKay. Brooks, K. W. (1979). Delphi technique: Expanding applications. North Central Association Quarterly, 54(3), 377-385. Case, B., Jorgensen, M., Zucker, S. (2004). Alignment in Educational Assessment. Pearson Education, Inc. Council of Chief State School Officers (CCSSO). (2002, September). Models for alignment analysis and assistance to states.

Washington, DC: Author. Retrieved May 30, 2017 from https://secure.wceruw.org/seconline/Reference/AlignmentModelsforStateAssist02.pdf

Coherent Curriculum Definition. (2014, March 3). Retrieved from https://www.edglossary.org/coherent-curriculum/. Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA:

Sage. Curriculum alignment. (2016, May 24). Retrieved from http://www/ibe.unesco.org/en/glossary -curriculum-

terminology/c/curriculum-alignment. Curriculum Development Council (CDC). (2001). Learning to learn – the way forward in curriculum. HKSAR Government:

Printing Department. Custer, R. L., Scarcella, J. A., & Stewart, B. R. (1999). The modified Delphi technique: A rotational modification. Journal of

Vocational and Technical Education, 15(2), 1-10. Cyphert, F. R., & Gant, W. L. (1971). The Delphi technique: A case study. Phi Delta Kappan, 52, 272-273. Dalkey, N. C. (1972). The Delphi method: An experimental study of group opinion. In N. C. Dalkey, N. C., & Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Management

Science, 9(3), 458-467. Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program planning. Glenview, IL: Scott,

Foresman, and Co. Department of Education (DepEd). (2016). K to 12 Araling Panlipunan Gabay Pangkurikulum, May 2016. Pasig City. Retrieved

March 5, 2018 from http://www.deped.gov.ph/sites/default/files/page/2017/AP%20CG!.pdf. Enacted curriculum. (2016, March 1). Retrieved from http://www.ibe.unesco.org/en/glossary-curriculum-terminology/i/enacted-

curriculum. English, F. W. (1992). Deciding what to teach and test: Developing, aligning, and auditing the curriculum. Newbury Park, CA:

Corwin Press. English, F. W. (2000). Deciding what to teach and test: Developing, aligning, and auditing the curriculum. San Francisco, CA:

Corwin Press. Fraser, S. & Bosanquet, A. (2006) The curriculum? That’s just a unit outline, isn’t it?, Studies in Higher Education, 31(3), 269-

284, DOI: 10.1080/03075070600680521


17

Gamoran, A., Porter, A. C., Smithson, LJ., & White, P. A. (1997). Upgrading high school mathematics instruction; Improving learning opportunities for low- achieving, low-income youth. Educational Evaluation and Policy Analysis, 19, 325-338.

Glatthorn, A. A. (1999). Curriculum alignment revisited. Journal of Curriculum and Supervision, 15, 26. Gosselin, D. (2017). Competencies and Learning Outcomes. InTeGrate. Retrieved from

https://serc.carleton.edu/integrate/programs/workforceprep/competencies_and_LO.html Hauenstein, A. D. (1998). A conceptual framework for educational objectives. A holistic approach to traditional taxonomies.

Lanham: University Press of America. Hsu, Chia-Chien & Sandford, Brian A. (2007). The Delphi Technique: Making Sense of Consensus. Practical Assessment

Research & Evaluation, 12(10). Available online: http://pareonline.net/getvn.asp?v=12&n=10 Impara, J. C. (2001). Alignment: One element of an assessment’s instructional unity. Paper presented at the 2001 annual meeting

of the National Council on Measurement in Education, Seattle, WA. Intended curriculum. (2016, June 8). Retrieved from http://www.ibe.unesco.org/en/glossary-curriculum-terminology/i/intended-

curriculum. International Association for Online Learning (iNACOL). (2011). What is competency education? CompetencyWorks. Retrieved

from https://www.competencyworks.org/wp-content/uploads/2014/09/CWorks-Understanding-Competency-Education.pdf Jacobs, J. M. (1996). Essential assessment criteria for physical education teacher education programs: A Delphi study.

Unpublished doctoral dissertation, West Virginia University, Morgantown. Judd, R. C. (1972). Use of Delphi methods in higher education. Technological Forecasting and Social Change, 4 (2), 173-186. Kaira, L. T. (2010). Using Item Mapping to Evaluate Alignment between Curriculum and Assessment. Dissertations. 318.

Retrieved from http://scholarworks.umass.edu/open_access_dissertations/318 Keohane, D. (2008). Curriculum Alignment in a Grade 9 Social Studies Exam. Unpublished study, Holy Spirit Catholic School

Division No. 4, Lethbridge, Alberta. Kerr, D. (1999). Citizenship education: An international comparison. International Review of Curriculum and Assessment

Frameworks Paper (4). London, UK: Qualifications and Curriculum Authority. Kesidou, S. & Roseman, J. E. (2002) How well do middle school science programs measure up? Findings from Project 2061’s

curriculum review. Journal of Research in Science Education, 39, 522 - 549. La Marca, P. M., Redfield, D., Winter, P. C., Bailey, A. & Despriet, L. (2000). State Standards and State Assessment Systems: A

guide to alignment. Washington, DC; Council of Chief State Officers. Linn, R. L. & Gronlund, N. E. (2000). Measurement and assessment in teaching (8th ed.). Upper Saddle River, NJ: Prentice Hall. Linstone HA. (1978) The Delphi technique. Handbook of Futures Research. Westport, CT: Greenwood; p. 271–300. Ludwig, B. G. (1994). Internationalizing Extension: An exploration of the characteristics evident in a state university Extension

system that achieves internationalization. Unpublished doctoral dissertation, The Ohio State University, Columbus. Ludwig, B. (1997). Predicting the future: Have you considered using the Delphi methodology? Journal of Extension, 35(5), 1-4.

Retrieved from http://www.joe.org/joe/1997october/tt2.html Marsh, C. (1996). Perspectives: Key concepts for understanding curriculum 1. London: Fullman Press.


18

Marsh, C., & Willis, P. (2003). Curriculum: Alternative approaches, ongoing issues (3rd ed.). Upper Saddle River, NJ: Merrill/Prentice Hall.

Martone, A. & Sireci, S. G. (2009). Evaluating alignment between curriculum, assessments, and instruction. Review of

Educational Research, 79(4), 1332 - 1361. McGehee, J. J., & Griffith, L. K. (2001). Large-scale assessments combined with curriculum alignment: Agents of Change. Theory

into Practice, 40(2), 137-144. McNamara, C. (1999) General guidelines for conducting interviews. Retrieved from

http://www.mapnp.org/library/evaluatn/intrview.htm Mordica, J., & Nicholson-Tosh, K. (2013). Curriculum alignment module. Champaign, IL: Office of Community College

Research and Leadership, University of Illinois at Urbana-Champaign. Retrieved from https://occrl.illinois.edu/docs/librariesprovider4/ptr/curriculum-alignment-module.pdf?sfvrsn=30b6bf89_9

Morse, J. (1991). On the evaluation of qualitative proposals. Qualitative Health Research, 1(2), 147-151. Moss, M. F. (1999). All students can learn: Effects of curriculum alignment on the mathematics achievement of third-grades

students. Paper presented at the Annual Meeting of the American Educational Research Association, Montreal, Quebec. Näsström, G. (2008). Alignment of standards and assessment: A theoretical and empirical study of methods for alignment. Umeå

University National Council for the Social Studies (NCSS) (2010). National Curriculum Standards for Social Studies: Chapter 2—The

Themes of Social Studies. Maryland. Retrieved from https://www.socialstudies.org/standards/strands Nikolov R., Shoikova E., & Kovatcheva E. (2014). Competence based framework for curriculum development. Zabukvite,

O’pismeneh, Sofia. Retrieved from https://unesco.unibit.bg/sites/default/files/CBFramework_TEMPUS-PICTET_Nikolov_Shoikova_Kovatcheva.pdf

No Child Left Behind Act of 2001, 20 U.S.C § 6311 et seq. (2001). Patton, M.Q. (2002). Qualitative evaluation and research methods. (3rd ed.) Thousand Oaks, CA: Sage. Porter, A. C. (2002). Measuring the content of instruction: Uses in research and practice. Educational Researcher, 31(7), 3-14. Porter, Andrew C. (2006). Handbook of Complementary Methods in Education Research. Retrieved from

https://www.routledgehandbooks.com/doi/10.4324/9780203874769.ch8 Porter, A. C., Floden, R., Freeman, D., Schmidt, W., and Schwille, J. (1988). Content determinants in elementary school

mathematics. In D. A. Grouws and T. J. Cooney (Eds.), Perspectives on research on effective mathematical teaching (pp. 96-113). Hillsdale, NJ: Lawrence Erlbaum Associates. (Also Research Series 179, East Lansing: Michigan State University, Institute for Research on Teaching.)

Porter, Andrew C. and Smithson, John L. (2001). Defining, Developing, and Using Curriculum Indicators. CPRE Research

Reports. Retrieved from http://repository.upenn.edu/cpre_researchreports/69 Print, M. (1993). Curriculum development and design (2nd Ed.). St Leonards: Australia: Allen & Unwin. Resnick, L. B., Rothman R., Slattery, J. B., & Vranek, J. L. (2003). Benchmarking and alignment of standards and testing.

Educational Assessment, 9(1 & 2), 1–27. Richmond, W. K. (1971). The school curriculum. London: Methuen.


19

Roach, A. T., Niebling, B. C., & Kurz, A. (2008). Evaluating the alignment among curriculum, instruction, and assessments: Implications and applications for research and practice. Psychology in the Schools, 45, 158-176.

Ross, A. (2001). Curriculum studies and critique. London: Falmer. Rothman, R., Slattery, J. B., Vranek, J. L., & Resnick, L. B. (2002). Benchmarking and alignment of standards and testing (CSE

Technical Report No. CSE-TR- 566). Los Angeles, CA: National Center for Research on Evaluation, Standards, and Student Testing.

Schwille, J. R., Porter, A. C., Belli, G., Floden, R., Freeman, D. J., Knappen, L. B., Kuhs, T. M., and Schmidt, W. H. (1983).

Teachers as policy brokers in the content of elementary school mathematics. In L. Shulman and G. Sykes (Eds.), Handbook on teaching and policy (pp. 370-391). New York: Longman. (Also Research Series No. 113, East Lansing, MI: Michigan State University, Institute for Research on Teaching, 1983.)

Scott, R. (1983). Curriculum Alignment as a Model for School Improvement. Seitz, P. (2014). Exploring and measuring curriculum alignment practices in the classroom: a mixed method approach. 147 pp.

Education and Research Archive. Retrieved from https://doi.org/10.7939/R30M36 Seitz, P. (2017). Curriculum Alignment Among the Intended, Enacted and Assessed Curricula for Grade 9 Mathematics. Journal

of the Canadian Association for Curriculum Studies (JCACS), 15(1), 2017, pp. 72-94. Retrieved from https://jcacs.journals.yorku.ca/index.php/jcacs/article/download/40286/36255

Skilbeck, M. (1998). School-based curriculum development. In Hargreaves, A. International handbook of educational change /

editors, andy hargreaves ... [et al.]. Dordrecht, The Netherlands: Kluwer Academic Publishers, 121-144. Slocum, N. (2005). Participatory Methods Toolkit Practitioner’s Manual Delphi: United Nations University – Comparative

Regional Integration Studies Squires, D. (2012). Curriculum alignment research suggests that alignment can improve student achievement. The Clearing

House: A Journal of Educational Strategies, Issues and Ideas, 85(4), 129-135. Taylor, R. E., & Judd, L. L. (1989). Delphi method applied to tourism. In S. Witt, & L. Moutinho, (Eds.). Tourism marketing and

management handbook. New York: Prentice Hall. Thangaratinam, S. & Redman, C.W.E. (2005). The Delphi technique. The Obstetrician & Gynaecologist;7:120–125 Tindal, G. (2005). Alignment of Alternate Assessments using the Webb System. Washington, DC; Council of Chief State Officers. Tindal, G., Cipoletti, B., & Almond, P. (2005). Alternate assessments: Evidence based on test content and alignment with

standards. Eugene, OR: University of Oregon Behavioral Research and Teaching. Tyler, R.W. (1949). Basic principles of curriculum and instruction. Chicago: University of Chicago Press. U.S. Department of Education. (2002). No Child Left Behind Act of 2001, Pub. L. No. 107–110. 115 Stat. 1425 (2002). Retrieved

from http://www.ed.gov/ policy/elsec/leg/esea02/107-110.pdf U.S. Department of Education. (2003, March 10). Standards and assessments: Non-regulatory guidance. Washington, DC:

Author. Urbiztondo, M.V. (2017). Development of a congruence index inventory for the assessment of curriculum alignment. Master’s

Thesis, University of the Philippines Diliman, Quezon City Walberg, H. J., “Productive teaching.” In Allan C. Ornstein, Teaching: Theory into Practice. Boston: Allyn and Bacon, 1995. Walker AM, Selfe J. (1996). The Delphi technique: a useful tool for the allied health researcher. British Journal of Therapy and

Rehabilitation; 3,677–80.


20

Waxman, H. C., & Huang, S. L. (1998). Classroom learning environments in urban elementary, middle, and high school. Learning

Environment Research, 1(1), 95-113. Webb, N. L. (1997a, April). Research monograph No. 6. Criteria for alignment of expectations and assessments in mathematics

and science education. Washington, DC: Council of Chief State School Officers. Webb, N. L. (1997b). Determining alignment of expectations and assessments in mathematics and science education. NISE Brief,

1(2). Webb, N. L. (1999). Research monograph No. 18. Alignment of science and mathematics standards and assessments in four

states. Washington, DC: Council of Chief State School Officers. Webb, N. L. (2002). Alignment study in language arts, mathematics, science, and social studies of state standards and

assessments for four states. Washington, DC: Council of Chief State of School Officers. Wiggins, G. & McTighe, J. (2001). Understanding by Design (2nd Ed.). Alexandria, VA: Association for Supervision and

Curriculum Development. Witkin, B. R., & Altschuld, J. W. (1995). Planning and conducting needs assessment: A practical guide. Thousand Oaks, CA:

Sage Publications, Inc.

evaluating the alignment of the intended and enacted school

Documents