Lesh & Clark (2000) - Formulating operational definitions of desired outcomes of instruction in mathematics and science education

Lesh, R., & Clarke, D. (2000). Formulating operational definitions of desired outcomes of instruction in mathematics and science education. In A. E. Kelly & R. A. Lesh (Eds.), Handbook of research design in mathematics and science education (pp. 113-149). Mahwah, NJ: Lawrence Erlbaum.

In science, some phenomena (e.g., neutrinos, black holes) may not be directly observable, but may be knowable by its effects (p.125). The authors argue that many aspects of learning and knowledge may not be directly observable either, but we can make inferences about what someone knows by what they can do. "For example, we may not know how to define what makes Granny a great cook; however, it still may be easy to identify situations that will elicit and reveal her capabilities, and it also may be easy to compare and assess alternative results that are produced." (p.140)

p127: Cognitive objectives function similarly to the ways cyclotrons, cloud chambers, and vats of heavy water are used in physics. That is, they are defined operationally by specifying: (a) situations that optimize the chances that the targeted construct will occur in an observable form; (b) observation tools that enable observers to sort out signal from noise in the results that occur; and (c) quality assessment criteria that allow meaningful comparisons to be made among alternative possibilities.

p130: In particular, in the case of conceptual systems that students develop during the solution of individual problem solving sessions: (i) model-eliciting activities put students in situations where they confront the need to produce a given type of construct, and where the products that they generate require them to reveal explicitly important characteristics of their underlying ways of thinking; (ii) ways of thinking sheets focus on ways of recognizing are describing the nature of the constructs that students produce; and (iii) guidelines for assessing the quality of students' work provide criteria that can be used to compare the usefulness of alternative ways of thinking.

p133: Three final characteristics should be mentioned that pertain to operational definitions involving the development of students, teachers, and programs. First, the development of these problem solvers tends to be highly interdependent. Second, when something (or someone) acts on anyone of these complex systems, they tend to act back. Third, researchers (as well as the instruments that they use) usually are integral parts of the systems that they are hoping to understand and explain.