Stiggins - Assessment Matrix of Targets and Methods

Stiggins, R.J. (2005). Student-Involved Assessment for Learning, 4th ed. (p. 65). Upper Saddle River, New Jersey: Prentice-Hall, Inc.

Resources:

AEA 267

PBS Teacher Line - Stiggins Chapter

Stiggins articles: Students view

Bena's version

Lesh & Clark (2000) - Formulating operational definitions of desired outcomes of instruction in mathematics and science education

Lesh, R., & Clarke, D. (2000). Formulating operational definitions of desired outcomes of instruction in mathematics and science education. In A. E. Kelly & R. A. Lesh (Eds.), Handbook of research design in mathematics and science education (pp. 113-149). Mahwah, NJ: Lawrence Erlbaum.

In science, some phenomena (e.g., neutrinos, black holes) may not be directly observable, but may be knowable by its effects (p.125). The authors argue that many aspects of learning and knowledge may not be directly observable either, but we can make inferences about what someone knows by what they can do. "For example, we may not know how to define what makes Granny a great cook; however, it still may be easy to identify situations that will elicit and reveal her capabilities, and it also may be easy to compare and assess alternative results that are produced." (p.140)

p127: Cognitive objectives function similarly to the ways cyclotrons, cloud chambers, and vats of heavy water are used in physics. That is, they are defined operationally by specifying: (a) situations that optimize the chances that the targeted construct will occur in an observable form; (b) observation tools that enable observers to sort out signal from noise in the results that occur; and (c) quality assessment criteria that allow meaningful comparisons to be made among alternative possibilities.

p130: In particular, in the case of conceptual systems that students develop during the solution of individual problem solving sessions: (i) model-eliciting activities put students in situations where they confront the need to produce a given type of construct, and where the products that they generate require them to reveal explicitly important characteristics of their underlying ways of thinking; (ii) ways of thinking sheets focus on ways of recognizing are describing the nature of the constructs that students produce; and (iii) guidelines for assessing the quality of students' work provide criteria that can be used to compare the usefulness of alternative ways of thinking.

p133: Three final characteristics should be mentioned that pertain to operational definitions involving the development of students, teachers, and programs. First, the development of these problem solvers tends to be highly interdependent. Second, when something (or someone) acts on anyone of these complex systems, they tend to act back. Third, researchers (as well as the instruments that they use) usually are integral parts of the systems that they are hoping to understand and explain. 

Binet and the history of intelligence testing

Cole, M. (1996). Cultural Psychology: A once and future discipline. Harvard University Press. Chapter 2 - pp 52-68

[p52] To begin with, Binet and Simon offered a definition of the quality they sought to test for: "It seems to us that in intelligence there is a fundamental faculty, the alteration or lack of which is of the utmost importance for practical life. This faculty is judgment, otherwise called good sense, practical sense, initiative, the faculty of adapting oneself to circumstances. To judge well, to comprehend well, to reason well, these are the essential activities of intelligence" (Binet and Simon, 1916, p. 43).

[p54] Although Binet and Simon specifically warned against the procedure, their test, and tests like it, began to be used as measures of overall aptitude for solving problems in general, rather than samples of problem-solving ability and knowledge that are important to formal education in particular. They also ignored Binet and Simon's warning about dealing with children from different cultural backgrounds.

[p57] Whatever IQ test performance is not related to, it most certainly is related to schooling.

Learning: from speculation to science

Bransford, J., Brown, A., Cocking, R. (2000). Learning: from speculation to science, Chapter 1 in How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.

Summary & Key Ideas

• One of the hallmarks of the new science of learning is its emphasis on learning with understanding.
• Humans are viewed as goal-directed agents who actively seek information.
• New knowledge must be constructed from existing knowledge -- THEREFORE teachers need to pay attention to the incomplete understandings, the false beliefs, and the naive renditions of concepts that learners bring with them to a given subject
• A common misconception regarding “constructivist” theories of knowing (that existing knowledge is used to build new knowledge) is that teachers should never tell students anything directly but, instead, should always allow them to construct knowledge for themselves. After people have first grappled with issues on their own, “teaching by telling” can work extremely well.
• It is important to help people take control of their own learning. Since understanding is viewed as important, people must learn to recognize when they understand and when they need more information. What strategies might they use to assess whether they understand someone else’s meaning? What kinds of evidence do they need in order to believe particular claims? How can they build their own theories of phenomena and test them effectively? [metacognition]

Key Findings from Research on Learning and Learners

• Students come to the classroom with preconceptions about how the world works. If their initial understanding is not engaged, they may fail to grasp the new concepts and information that are taught, or they may learn them for purposes of a test but revert to their preconceptions outside the classroom.
• To develop competence in an area of inquiry, students must: (a) have a deep foundation of factual knowledge, (b) understand facts and ideas in the context of a conceptual framework, and (c) organize knowledge in ways that facilitate retrieval and application.
• A “metacognitive” approach to instruction can help students learn to take control of their own learning by defining learning goals and monitoring their progress in achieving them.

Implications for Teaching

• Teachers must draw out and work with the preexisting understandings that their students bring with them.
• Teachers must teach some subject matter in depth, providing many examples in which the same concept is at work and providing a firm foundation of factual knowledge
• The teaching of metacognitive skills should be integrated into the curriculum in a variety of subject areas.

Other interesting ideas

• Learning is influenced in fundamental ways by the context in which it takes place. A community-centered approach requires the development of norms for the classroom and school, as well as connections to the outside world, that support core learning values

Critique of PD programs for teachers. Professional development programs for teachers, for example, frequently:

• Are not learner centered. Rather than ask teachers where they need help, they are simply expected to attend prearranged workshops.
• Are not knowledge centered. Teachers may simply be introduced to a new technique (like cooperative learning) without being given the opportunity to understand why, when, where, and how it might be valuable to them. Especially important is the need to integrate the structure of activities with the content of the curriculum that is taught.
• Are not assessment centered. In order for teachers to change their practices, they need opportunities to try things out in their classrooms and then receive feedback. Most professional development opportunities do not provide such feedback. Moreover, they tend to focus on change in teaching practice as the goal, but they neglect to develop in teachers the capacity to judge successful transfer of the technique to the classroom or its effects on student achievement.
• Are not community centered. Many professional development opportunities are conducted in isolation. Opportunities for continued contact and support as teachers incorporate new ideas into their teaching are limited, yet the rapid spread of Internet access provides a ready means of maintaining such contact if appropriately designed tools and services are available.