Herman (1998) - Promoting pedagogical reasoning as preservice teachers analyze case vignettes

Herman, W.E. (1998). Promoting pedagogical reasoning as preservice teachers analyze case vignettes. Journal of Teacher Education , 49, 391-397

P.H.D. = pedagogical heuristics device

The amount of class time to explore the detail of the P.H.D. dictated the need for brevity in the classroom stimulus. Written case vignettes were brief (1-2 pages) and offered critical incidents that could lead to depth of analysis and teacher interventions. Many excellent sources of case studies exist (e.g., Kowalski, Weaver, & Henson, 1990; Silverman et al., 1994).

This heuristic device is a general set of questions and tasks guiding future teachers in the process of solving educational problems. Shavelson and Stern (1981) defined heuristics as implicit rules that people are unaware of and use in complex tasks in order to select information, classify objects or persons, or revise their knowledge (p.469). Students learned to use this P.H.D. to interpret case vignettes depicting classroom situations, link theory to the educational context, and develop teacher action plans.

They began by analyzing the situation in terms of such elements as historical time period, emotional states, quality of the relationships, verbal/nonverbal messages, motivations, defenses, and social/multicultural elements.

Next, students were asked to link five theoretical constructs to the scene on the basis of their potential to add understanding to the teaching and learning aspects of the situation.

Students then were asked to describe each selected construct, then generate 10 possible teacher responses along with possible student reactions to these responses.

Finally, students generated and evaluated a five-point action plan based upon internal/external criteria, why it might work or fail, and moral/ethical/ political considerations.

Social interaction and individual understanding in a community of learners: The influence of Piaget and Vygotsky

Brown, A. L., Metz, K. M. & Campione, J. C. (1996). Social interaction and individual understanding in a community of learners: The influence of Piaget and Vygotsky. In A. Tryphon & J. Vonèche (Eds.), Piaget- Vygotsky: The social genesis of thought. (pp. 145-170). Mahwah, New Jersey: Lawrence Erlbaum.

Piaget and Vygotsky have more in common than is usually supposed.

Piaget's Influence

What can children be reasonably expected to learn and understand?

Piaget argued that the development of logical thought is enhanced by the need to defend one's ideas to actual or imagined audiences.

Grade school child as a scientist (or theorist) concept:

children of this age are able to identify variables, determine cause, and refine theories

children's reasoning can be much more sophisticated in familiar situations and well developed knowledge

children's thinking often reflect a natural inductive logic

Vygotsky's Influence

Participant structures:
1. Reciprical teaching
2. Jigsaw
3. Guided writing

Main principles
1. classrooms invoke multiple zones of proximal development
2. a community of academic and scientific discourse is developed
3. meaning is negotiated and refined
4. ideas are seeded and appropriated; and
5. common knowledge and distributed expertise are both essential

Treagust (2006). Conceptual change as a viable approach to understanding student learning in science

Treagust, D. F. (2006). Conceptual change as a viable approach to understanding student learning in science. In Kenneth Tobin (Ed.), Teaching and Learning Science: A Handbook, 25-32. Lanham, Maryland: Rowman & Littlefield.

The best-known conceptual change model in science education, based on students' epistemologies-examining how students think about their world originated with George Posner, Kenneth Strike, Peter Hewson, and William Gertzog (1982).

In this conceptual change model, student dissatisfaction with a prior conception was believed to initiate dramatic or revolutionary conceptual change and was embedded in constructivist epistemological views with an emphasis on the individual's conceptions and his or her conceptual development.

If the learner was dissatisfied with his or her prior conception and an available replacement conception was intelligible, plausible, and/or fruitful, accommodation of the new conception may follow.

An intelligible conception is sensible if it is noncontradictory and its meaning is understood by the student; plausible means that in addition to the student knowing what the conception means, he or she finds the conception believable; and the conception is fruitful if it helps the learner solve other problems or suggests new research directions.

The extent to which the conception meets these three conditions is termed the status of a learner's conception. Resultant conceptual changes may be permanent, temporary, or too tenuous to detect.

Halliday - Learning Language, Learning through Language, Learning about Language.

Halliday, M.A.K. (2004). Three Aspects of Children's Language Development: Learning Language, Learning through Language, Learning about Language. In J.J. Webster (ed.), The Language of Early Childhood: M.A.K. Halliday, pp 308-326, Ch. 14. New York: Continuum - ISBN - 0826488250

Learning Language (p. 308) - a child starts learning language from the moment he is born. In fact, the baby has started learning language before he was born ... from birth onwards, he is actively involved in communication, exchanging signals with the other human begins around him.

Learning through Language (p. 317) - refers to language in the construction of reality: how we use language to build up a picture of the world in which we live ... the part played by language in shaping and transmitting the world view of each and every human culture

Learning about Language (p. 322)- coming to understand the nature and functions of language itself

Google Books

First published in Yetta M. Goodman, Myna M. Hausser and Dorothy S. Strickland (eds.) Oral and Written Langauge Development: Impact on Schools. International Reading Association & National Council of Teachers of English: Proceedings from the 1979 and 1980 IMPACT Conferences. pp.7-19.

Also see Pauline Gibbon's Scaffolding Language, Scaffolding Learning (2002), Ch. 7, pp. 118-139

Lesh & Clarke (2000) Formulating operational definitions of desired outcomes of instruction in mathematics and science education

Lesh, R.A. & Clarke, D. (2000). Formulating operational definitions of desired outcomes of instruction in mathematics and science education. In A.E. Kelly & R.A. Lesh (eds.) Handbook of Research Design in Mathematics and Science Education, pp 113-49. Mahwah, NJ: Lawrence Erlbaum Associates

p120: Later in this chapter, other explanations will be given about why teaching to tests, over a long period of time, tends to be a losing strategy. For now, a brief explanation is that: (a) students soon forget disorganized lists of facts and skills; (b) when facts and skills are "mastered" one-at-a-time and in isolation, students may never learn when to chose which one to use in particular situations; and (c) when instruction emphasizes only facts and skills, or content-independent problems solving processes, other exceedingly important goals of instruction are ignored.

p121: high scores on tests often are treated as if they went beyond being indicators of achievement toward actually being the goals of instruction. [JC: see Labaree and his discussion of credentialism]
p122: In past stages of history, educators have tended to think about the mind (and about the nature of mathematical knowledge) as if it were similar to the most sophisticated technology of the preceding age. For example, as civilizations evolved from the industrial revolution through the electronics revolution to the current age of biotechnologies, educators have shifted from machine metaphors (based on hardware) to computer metaphors (based on software) to organic metaphors (based on wetware of the type that characterizes the processes of the human brain). Yet, in the area of assessment, simple input-output models continue to dominate that are based on machine metaphors.

p122: (NCTM, 1989)
In spite of the best intentions of developers and implementors. it was unreasonable to expect that new products or programs would be used as intended in most schools and classrooms. The reason for this is that public schools as they now operate are integrated social systems. Tinkering with parts. such as changing textbooks or the number of required courses, fails to change other components of the system. The traditions of the system force new products to be used in old ways. Current educational practice is based on a coherent set of ideas about goals. knowledge. work. and technology that came from a set of "scientific management" principks growing out of the industrial revolution of the past century. These ideas about schooling need to be challenged and replaced with an equally coherent set of practices in light of the economic and social revolution in which we are now engaged. Current school mathematics operates within a coherent system; reform will happen only if an equally coherent system replaces it. (National Council of Teachers of Mathematics. 1989)

p126: Behavioral Objectives Involve Three Parts
Given {specified conditions} the student will exhibit {specified behaviors} with identifiable quality {perhaps specified as percents correct on relevant samples of tasks, or perhaps specified in terms of a correspondence with certain criteria for excellence}.

In mathematics and science education, the main problem with behavioral objectives is that not all forms of learning consist of rules (facts, skills, procedures); and, if attempts are made to reduce more complex conceptual systems to checklists of rules, the following sorts of difficulties arise.

For instance, it may be true that a great artist (or a great athlete) should be able to perform well on certain basic drills and exercises (calisthenics); nonetheless, a program of instruction (or assessment) that focuses on nothing more than these checklists of basic skills is unlikely to provide adequate approaches to excellence.

p127: Cognitive objectives function similarly to the ways cyclotrons, cloud chambers, and vats of heavy water are used in physics. That is, they are defined operationally by specifying: (a) situations that optimize the chances that the targeted construct will occur in an observable form; (b) observation tools that enable observers to sort out signal from noise in the results that occur; and (c) quality assessment criteria that allow meaningful comparisons to be made among alternative possibilities.

p130: In particular, in the case of conceptual systems that students develop during the solution of individual problem solving sessions: (i) model-eliciting activities put students in situations where they confront the need to produce a given type of construct, and where the products that they generate require them to reveal explicitly important characteristics of their undedying ways of thinking; (ii) ways of thinking sheets focus on ways of recognizing arxi describing the nature of the constructs that students produce; and (iii) guidelines for assessing the quality of students' work provide criteria that can be used to compare the usefulness of alternative ways of thinking.

p133: Three final characteristics should be mentioned that pertain to operational definitions involving the development of students, teachers, and programs. First, the development of these problem solvers tends to be highly interdependent. Second, when something (or someone) acts on anyone of these complex systems, they tend to act back. Third, researchers (as well as the instruments that they use) usually are integral parts of the systems that they are hoping to understand and explain.

p.135-6: Examples of such teacher-level activities include generating:
(a) observation sheets that colleagues could use to make significant observations about students as they are working in groups;
(b) "ways of thinking sheets" that colleagues could use to give feedback to students about the strengths or weaknesses of their work; and
(c) quality assessment procedures that colleagues could use to lead discussions with students aimed at assessing the quality of the products that students produce.

For the purposes of this chapter, three of the most important characteristics of multitiered teaching experiments are that:
1. They use formative feedback and consensus-building to provide conditions that make it possible for students, teachers, and/or programs to develop in directions that are continually "better" without basing the next steps on preconceived notions of "best."
2. They emphasize the use of self-documenting activities that encourage students, teachers, and/or programs to learn while simultaneously producing trails of documentation that reveal important characteristics about the nature of what is being learned.
3. The preceding trends allow inferences to be made about future developments that are likely to occur.

p.136: One of the most important assumptions underlying the teacher level of multi-tiered teaching experiments is that, to improve teachers' teaching practices, it is not enough to ensure familiarity with a checklist of behavioral objectives; teachers also need to develop productive ways thinking about their teaching experiences. In particular, teachers need to develop productive ways of thinking about their students' learning and problem solving experiences.

p.136: Cognitively guided instruction is a name that's been given to an approach to teacher development that focuses on helping teachers become "reflective practitioners" by becoming familiar with new insights about the nature of students' developing mathematical knowledge (T. Carpenter & Fennema, 1992).

p.137: rather than telling teachers about their students' ways of thinking, thought-revealing activities are used so that teachers can make firsthand observations about their students' ways of thinking.

p.138: in studies of developing expertise of teachers (Lesh & Kelly, 1998), it is not necessary for a given description of expertise to be locked in at the beginning of a study (and used as the basis for a pretest-posttest design). Instead, increasingly sophisticated descriptions can be refined and documented gradually over the course of the study; and, at the end of the study, the validity of the description can be based on the trajectory of development that is revealed. In fact, as teachers, programs, and schools develop, their notions of excellence in teaching are primary factors that change. Therefore, if their progress continues to be measured using conceptions of excellence that existed at the beginning of a study, this practice tends to have significant negative influences on development.

p.139: An alternative to conformance models for curriculum change might be called planned experimentation.

p.140 Contexts that elicit complex performances don't necessarily require the relevant participants to reveal observable ways of thinking; and, they also don't necessarily provide useful tools for comparing or assessing the quality of competing systems. Nonetheless, it often is not difficult to identify situations that require the relevant systems to be elicited and revealed, and it often is not difficult to identify ways to compare and assess the results that are produced. For example, we may not know how to define what makes Granny a great cook; however, it still may be easy to identify situations that will elicit and reveal her capabilities, and it also may be easy to compare and assess alternative results that are produced.

p.141: some important principles to keep in mind include the following:
1. Achievement usually needs to be assessed using something more than brief tests that reduce expertise to simplistic lists of condition-action rules.
2. Students' relevant products or performances usually should include more than those that can be interpreted and assessed easily by a machine.
3. Emphasis needs to shift beyond superficial coverage of a large number of small tasks to the comprehensive treatment of a small number of big ideas.
4. Quality ratings should not ignore the conditions under which complex performances occur, and complex profiles should not be collapsed into simplistic scores on a scale that recognizes only a single dimension along which progress can be made.

p.144: three-fold character of model-eliciting activities: situations, products, and quality assessment criteria This structure focuses attention (as classroom practitioners or as educational researchers, for instance) on activities that create situations conducive to the constructs to be studied, requires products or observation tools through which the construct is made manifest, and produces these products in a form amenable to assessment against specified criteria for quality.

p.144: perhaps the most contentious of the suggestions put forward in this chapter is advocacy of the use of assessment to change the observed system purposefully toward the desired outcomes. In this, the authors anticipate the development of assessment systems that both reveal valued constructs and deliberately prompt adaptations within the systems studied in directions advantageous to the system or to the individual whose performances are assessed.

Tochon - From video cases to video pedagogy

Tochon, F.V. (2007). From video cases to video pedagogy: A framework for video feedback and reflection in pedagogical research praxis. In R. Goldman, R. Pea, B. Barron, & S.J. Derry (Eds.), Video Research in the Learning Sciences, pp. 53-65. Mahwah, NJ: Routledge

p. 56:
Video pedagogy involves the art of choosing the right framework for shared reflection. This constitutes a basic principle of video study groups: The framework of reflection specifies the point of impact of video feedback. The way video is used is closely dependent on the framework chosen by the group. It must be concerted and well defined.

What frameworks are available for video study groups? Several have been found especially helpful to shared reflection. 

• The mastery framework orients activities in light of the desired results of the task to be performed. 
• The psychocognitive framework emphasizes the conceptual structures of information and strategies for learning. 
• In the sociocognitive framework, the process of bringing thought to awareness is linked to authentic experiences that challenge widely held notions. 
• The narrative framework is the foundation for autobiographical or personal approaches to human experience. 
• The critical framework functions in a social and participatory perspective. It proposes empowerment over the act of learning and an examination of those aspects of interaction that turn education into a process of either oppression or liberation. 
• The pragmatic framework is focused on the language of practice. It explicates practical discursive arguments and intentions related to situations.

p. 55:
Following Croue (1997), a typology of video cases could be raised:

• (a) Cases that dealt with the nature of a problem were related to decision making, to assessment or feedback; 
• (b) Cases that dealt with the range of the problem presented a problem or showed the interconnection between various problems; 
• (c) Cases that took a story format presented a story line, with episodes, props, and portraits.

Croue. C. (1997). Introduction a la methode des cas [An Introduction to the case method]. Paris: Gaetan Morin Europe.

Cochran-Smith & Lytle (1999): Relationships of Knowledge and Practice: Teacher Learning in Communities

Cochran-Smith, M., Lytle, S.L (1999). Relationships of Knowledge and Practice: Teacher Learning in Communities. Review of Research in Education, Jan 1999; vol. 24: pp. 249-305. Retrieved from http://rre.sagepub.com/cgi/content/short/24/1/249

Three conceptions of teacher knowledge and learning:

knowledge for practice: formal knowledge and theory (including codifications of the so-called wisdom of practice) for teachers to use in order to improve practice

knowledge in practice: Here it is assumed that teachers learn when they have opportunities to probe the knowledge embedded in the work of expert teachers and/or to deepen their own knowledge and expertise as makers of wise judgments and designers of rich learning interactions in the classroom.

knowledge of practice: it is assumed that the knowledge teachers need to teach well is generated when teachers treat their own classrooms and schools as sites for intentional investigation at the same time that they treat the knowledge and theory produced by others as generative material for interrogation and interpretation

FIGURE 1
TEACHER LEARNING: A CONCEPTUAL FRAMEWORK

KNOWLEDGE-PRACTICE RELATIONSHIP
What is understood or assumed to be the relationship of knowledge and practice? What is assumed about how "knowing more" and "teaching better" are connected?

IMAGES OF KNOWLEDGE
What knowledge are teachers assumed to need in order to "teach better"? What are the domains, sources, or forms of that knowledge? Who generates that knowledge? Who evaluates and interprets that knowledge?

IMAGES OF TEACHERS, TEACHING, AND PROFESSIONAL PRACTICE
What is assumed about the nature of the activity of teaching? What is included in the idea of "practice"? What are assumed to be the primary roles of teachers in and out of classrooms? What is the relationship of teachers' work in and out of classrooms?

IMAGES OF TEACHER LEARNINGA ND TEACHERS' ROLES IN EDUCATIONAL CHANGE
What is assumed about the roles teachers and teacher learning play in educational change? What are assumed to be the intellectual, social, and organizational contexts that support teacher learning? What is the role of communities, collaboratives, and/or other collectives in these?

CURRENT INITIATIVES
What are current initiatives in teacher education, professional development and/ or teacher assessment that are based on these images?

Shulman’s Model of Pedagogical Reasoning and Action (1987)

Shulman’s Model of Pedagogical Reasoning and Action (1987)

Comprehension
Of purposes, subject matter structures, ideas within and outside the discipline

Transformation

Preparation: critical Interpretation and analysis of texts, structuring and segmenting, development of a curricular repertoire, and clarification of purposes

Representation: use of a representational repertoire which includes analogies, metaphors, examples, demonstrations, explanations, and so forth

Selection: choice from among an instructional repertoire which includes modes of teaching, organizing, managing, and arranging

Adaptation and Tailoring to Student Characteristics: consideration of conceptions, preconceptions, misconceptions, and difficulties, language, culture, and motivations, social class, gender, age, ability, aptitude, interests, self concepts, and attention

Instruction

Management, presentations, Interactions, group work, discipline, humor, questioning, and other aspects of active teaching, discovery or Inquiry Instruction, and the observable forms of classroom teaching

Evaluation
Checking for student understanding during Interactive teaching
Testing student understanding at the end of lessons or units
Evaluating one's own performance, and adjusting for experiences

Reflection
Reviewing, reconstructing, reenacting and critically analyzing one's own and the class's performance, and grounding explanations in evidence

New Comprehensions
Of purposes, subject matter, students, teaching, and self
Consolidation of new understandings, and learnings from experience

Shulman, L. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1-22.

Productive pedagogy framework

Gore, J.M., Griffiths, T. & Ladwig, J.G. (2004). Towards better teaching: productive pedagogy as a framework for teacher education. Teaching and Teacher Education, 20, 375–387.

Productive pedagogy (PP) has four dimensions:

1. Intellectual quality
2. Relevance
3. Supportive classroom environment
4. Recognition of difference

More broadly, PP principles challenge conventional understandings about what is important and what should be emphasised in teacher education programs. It suggests a re-thinking ofwhat is offered and what is valued. In particular, the principles of PP require teacher educators to address:
1. The overemphasis on classroom environments and processes rather than on substance and purposes.
2. The relationships between foundational studies, curriculum studies and field experiences which are currently insuffficiently connected.
3. The purpose and structure of field experiences which centre too often on practising teaching techniques with relatively little concern for what is being taught and the quality of learning produced.
4. The focus on student management relative to student learning, which mistakenly assumes that management should be addressed first and separately.
5. The emphasis on syllabus content and constraints of the formal curriculum relative to identifying central concepts and producing depth of understanding.

---

Productive pedagogy dimensions, items and key questions addressed

Intellectual quality
Higher order thinking: Are higher order thinking and critical analysis occurring?
Deep knowledge: Does the lesson cover operational fields in any depth, detail or level ofspecificity?
Deep understanding: Do the work and response ofthe students provide evidence ofunderstand ing ofconcepts or ideas?
Substantive conversation: Does classroom talk break out ofthe initiate/respond/evaluate pattern and lead to sustained dialogue between students, and between teachers and students?
Knowledge problematic: Are students critiquing and second-guessing texts, ideas and knowledge?
Metalanguage: Are aspects oflanguage, grammar, and technical vocabulary being foregrounded?

Relevance
Knowledge integration: Does the lesson range across diverse fields, disciplines and paradigms?
Background knowledge: Is there an attempt to connect with students’ background knowledge?
Connectedness to the world: Do lessons and the assigned work have any resemblance or connection to real life contexts?
Problem based curriculum: Is there a focus on identifying and solving intellectual and/or real-world problems?

Supportive classroom environment
Student control: Do students have any say in the pace, direction or outcome ofthe lesson?
Social support: Is the classroom a socially supportive, positive environment?
Engagement: Are students engaged and on-task?
Explicit criteria: Are criteria for student performance made explicit?
Self-regulation: Is the direction of student behaviour implicit and self-regulatory or explicit?

Recognition of difference
Cultural knowledges: Are diverse cultural knowledges brought into play?
Inclusivity: Are deliberate attempts made to increase the participation of all students of different backgrounds?
Narrative: Is the teaching principally narrative, or is it expository?
Group identity: Does teaching build a sense of community and identity?
Citizenship: Are attempts made to foster active citizenship?

Case analysis framework - McNergney, Herbert & Ford (1994)

Sudzina, M. (1999). Case study applications for teacher education: Cases of teaching and learning in the content areas. Boston: Allyn & Bacon.

Sudzina (1999), p. 16-17:

Case analysis framework
1. Identifying the facts and issues in a case
2. Identifying the perspectives and values
3. Identifying the professional knowledge
4. Formulating action
5. Considering the consequences of actions

---

Cooperation and Competition in Case-Based Teacher Education
Robert F. McNergney, Joanne M. Herbert, and Rudolph E. Ford
Journal of Teacher Education, November 1994; vol. 45, 5: pp. 339-345.

We characterize reflection, or professional thinking, in terms of five steps: perceiving problems and opportunities, recognizing values that drive actions, applying knowledge, taking action, and examining consequences (McNergney & Medley, 1984; McNergney, Herbert, & Ford, 1993). Conceptually, the five steps are sequential. An instructor encourages the development of this kind of reflection via cases by drawing attention to:

• facts and issues in a case;
• perspectives of the actors (e.g., teacher, students, parents, principal) or the values underlying actions individuals take in the case;
• professional knowledge born of practice, theory, and research relevant to problems in the case;
• projected teaching actions; and
• likely consequences of projected actions.

Instructors trigger considerations of these factors in students by asking questions and offering their own observations.

In practice, the steps are not necessarily conceptually discrete, and they frequently do not proceed in linear fashion. For example, knowledge considerations may influence identification of issues; discussion may move back and forth between these steps. And consequences or desired end results might overshadow all other considerations and serve as springboards to discussion of other factors. While an instructor can recognize a path from one end of the reflective process to the other, she or he might not be able to trace a clean, straight line from beginning to end. We encourage teachers to reflect on events in a case as they might think about events in real life, but to do so with attention to the five factors.

McNergney, R. F., Herbert, J. M., and Ford, R. E. (1994). Cooperation and Competition in Case-Based Teacher Education. Journal of Teacher Education, 45(5), 339-345.

Policies That Support Professional Development

Policies That Support Professional Development in an Era of Reform
Darling-Hammond, L., & McLaughlin, M. W. (1995). Phi Delta Kappan, 76(8), 642-645

The vision of practice that underlies the nation's reform agenda requires most teachers to rethink their own practice, to construct new classroom roles and expectations about student outcomes, and to teach in ways they have never taught before - and probably never experienced as students (Nelson & Hammerman, 1995).

Reformers of all stripes press for an agenda of fundamental change in the ways teachers teach and students learn. They envision schools in which students learn to think creatively and deeply, in which teachers' ongoing learning forms the core of professional activities, and in which students and teachers alike value knowing why and how to learn.(FN34)

These visions and expectations for practice assume fundamental changes in education policies in order to enable teachers to make the challenging and sometimes painful changes required of them. Yet these necessary shifts in policy have only begun.

---

Nelson, B.S. & Hammerman, J.M. (1996). Reconceptualizing teaching : moving toward the creation of intellectual communities of students, teachers, and teacher educators. In Milbrey W. McLaughlin and Ida Oberman (eds.), Teacher learning: new policies, new practices. New York: Teachers College Press

FN34: Nelson and Hammerman, op. cit.; Beverly Falk, "Teaching the Way Children Learn," in McLaughlin and Oberman, op. cit.; and Martin G. Brooks and Jacqueline Grennon Brooks, "Constructivism and School Reform," in McLaughlin and Oberman, op. cit.

Crooks, Kane & Cohen (2008) - Threats to the valid use of assessments

Crooks, T. J., Kane, M. T., & Cohen, A. S. (2008). Threats to the valid use of assessments. In H. Wynne (Ed.) Student assessment and testing: Vol. 2 (Chapter 21, pp. 151-171). Thousand Oaks, CA: Sage.

Main points:
1) Validity is an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores or other modes of assessment (Messick, 1989, p. 13) [p. 150]
2) Assessment can be depicted as a chain of eight linked stages (administration, scoring, aggregation, generalization, extrapolation, evaluation, decision and impact) which form an assessment system
3) Problems and issues in any of the links can threaten the confidence in an assessment system, and undermine any inferences and claims from this system
4) Validity estimation relies heavily on human judgment and is therefore harder to carry out, report and defend. [p. 150]
5) High reliability is necessary but not sufficient for high validity. Some degree of
reliability is essential for validity. Reliability establishes an upper limit for validity.

Theoretical Framework
Concept of validity argument: interpretations of assessments of performances involve a linked series of inferences and assumptions (Kane 1992; Shepard, 1993; Cronbach, 1988). If the inferences and assumptions can be identified, the plausibility can be examined by logical and empirical means, and the importance of each can be debated. [p.151]

Assessment validation model
Crooks, Kane and Cohen’s (2008) assessment validation model depicts assessment as involving eight linked stages:
(1) Administration of assessment tasks to the student.
(2) Scoring of the student’s performances on the tasks.
(3) Aggregation of the scores on individual tasks to produce one or more combined scores.
(4) Generalization from the particular tasks included in a combined score to the whole domain of similar tasks (the assessed domain).
(5) Extrapolation from the assessed domain to a target domain containing all tasks relevant to the proposed interpretation.
(6) Evaluation of the student’s performance, forming judgments.
(7) Decision on actions to be taken in light of the judgments.
(8) Impact on the student and other participants arising from the assessment processes, interpretations and decisions. [p. 153]

Gipps (2008) - Socio-cultural aspects of assessment

Gipps, C. (2008). Socio-cultural aspects of assessment. In H. Wynne (Ed.) Student assessment and testing: Vol. 1 (Chapter 8, pp. 252-291). Thousand Oaks, CA: Sage.

MAIN POINTS

1.     At all levels, assessment is a social activity and that we can understand it only by taking account of the social, cultural, economic, and political contexts in which it operates. (p. 252)

2.     Assessment plays an important role in cultural and social reproduction, in allocating educational and economic opportunities, and more recently, to control curriculum and teaching. (p. 264)

3.     Changes in assessment practice and design reflect changes in world view, a resulting change in epistemology, and new understandings of learning. (p. 273)

4.     There are complex interactions among students, teachers, and assessment. (p. 284)

5.     When designing assessment, there are trade-offs between reliability, validity, and assessment of higher order thinking skills. (p. 283)

6.     Theories about intelligence and learning have implications for assessment design (p.272)

7.     Although new approaches to assessment have the promise of being more equitable, performance assessments on their own will not enhance equity. (p. 283)

CLAIMS , EVIDENCE & REASONING

1.     CLAIM: Although IQ testing, objective testing, and external examinations were seen originally as equitable tools for selection and certification purposes, a sociological critique calls this into question. (p. 260)

EVIDENCE & REASONING:

a.     Performance at school may be affected by social and cultural background factors. Among these factors are poverty, poor resources at home and/or at school, absenteeism owing to work or domestic duties, mismatch between the language and culture of the home and the school, gender bias, and ethnic discrimination. As a result, examinations may be biased, and furthermore, because of their role in certification, they may institutionalize and legitimate social stratification. (p.361)

b.    Cultural capital argument: children from lower social groups are not less intelligent or less academically capable, but children from middle-class homes are better able to do well at school because of the correspondence of cultural factors between home and school. As a result, examinations have a legitimating role in that they allow the ruling classes to legitimate the power and prestige they already have. (p.361) [see Bourdieu & Passeron (1976)]

2.     CLAIM: It is possible to run a national assessment program that includes high-quality examinations and some performance assessment, and it is possible to design an assessment program with different features and purposes at different levels of the school system. (p.283)

EVIDENCE & REASONING: experience in England in the 1990s (Stobart & Gipps, 1997; James & Gipps, 1998) provides existence proofs that it is possible to implement new assessment systems at scale.

3.     CLAIM: Although performance assessment and evaluation of culturally sensitive classroom-based learning have the potential to foster multicultural inclusion and facilitate enhanced learning, performance assessment on its own will not enhance equity. (p. 283)

EVIDENCE & REASONING: Consideration must still be given to students' opportunity to learn (Linn, 1993), the knowledge and language demands of the task (Baker & O'Neil, 1995), and the criteria used for scoring (Linn, Baker, & Dunbar, 1991). Clearly, as with traditional forms of assessment, questions of fairness arise in the selection of tasks and in the scoring of responses. Furthermore, the more informal and open-ended such assessment becomes, the greater the reliance on the judgment of the teacher/assessor. Here we come again to the issue of power and control, a theme of this chapter. Alternative forms of assessment do not, of themselves, alter power relationships and cultural dominance in the classroom. (p.283)

FRAMEWORKS

Psychometric theory: Psychometric theory developed originally from work on intelligence and intelligence testing. The underlying notion was that intelligence was innate and fixed in the way that other inherited characteristics are, such as skin color. Intelligence could therefore be measured (since, like other characteristics, it was observable), and, on the basis of the outcome, individuals could be assigned to streams, groups, or schools that were appropriate to their intelligence (or "ability," as it came to be seen). … With the psychometric model comes an assumption of the primacy of technical issues, notably standardization and reliability (Goldstein, 1996). (p.263)

Constructivist learning theory: students learn by actively making sense of new knowledge, making meaning from it (Iran-Nejad, 1995), and mapping it into their existing knowledge map or schema. Shepard (1991) notes that "contemporary cognitive psychology has built on the very old idea that things are easier to learn if they make sense." (p.271)

Sociocultural learning theory: Socioculturalist assume human agency in the process of coming to know, but socioculturalists further argue that meaning derived from interactions is not exclusively a product of the person acting. They view the individual engaged in relational activities with others. Building on Vygotsky's arguments about the importance of interaction with more knowledgeable others and the role of society in providing a framework for the child's learning, sociocultural theorists thus describe learning in terms of apprenticeship (e.g., Brown et al., 1993; Glaser, 1990; Rogoff, 1990), legitimate peripheral participation (Lave & Wenger, 1991), or negotiation of meaning in the construction zone (Newman, Griffin, & Cole, 1989). (p. 271)

Tittle’s (1994) framework for an educational psychology of assessment: there are three dimensions: the epistemology and theories involved (both general and in relation to subject matter); the interpreter and user, whose presence, characteristics, needs, and values must be brought into the frame; and the characteristics of the assessment itself.

KEEPSAKES

1. Testing is now being used to control curriculum and teaching. (p. 283)
2. Developments in cognition and learning are telling us to assess more broadly, in context, and in depth. This requires methods of assessment that do not lend themselves readily to traditional reliability, highlighting the tension between types and purposes of assessment. (p. 283)
3. From an interpretivist viewpoint, it is important to acknowledge the complexity of interactions among students, teachers, and assessment. Factors such as students' perceptions of how testing affects them (Herman et al., 1997), student and teacher confidence in the veracity of test results, and differences in student and teacher perceptions of the goals of assessment all need to be considered. (p. 284)
4. We need to bring out into the open the nature of the power relationship in teaching and assessment and point out the possibility of reconstructing this relationship. Perhaps most important, we need to encourage teachers to bring pupils into the process of assessment, in order to recognize their social and cultural background, and into self-assessment, in order to develop their evaluative and metacognitive skills. (p.286)
5. A key direction for the future lies in the development of teachers' classroom assessment skills. It is evident from this chapter that some teachers are operating in collaborative, constructivist ways supported by portfolio work, for example, or as evidenced by their feedback to learners. Such practice is not common but clearly can become part of the teacher's repertoire. This implies the continued development of new assessment strategies for use by teachers, involving group and interactive assessment and interview and portfolio approaches. It will involve extending teachers' skills in observation and questioning while making them aware of social and cultural influences on the assessment process. (p. 286)

Gutiérrez & Orellana (2006) - The "Problem" of English Learners: Constructing Genres of Difference

Gutiérrez, K. & Orellana, M.F. (2006). The "Problem" of English Learners: Constructing Genres of Difference. Research in the Teaching of English, 40, 502-507.

Key terms:

selective exemplification: Taking a "slice of life" of English Learners in a way that does not show complexity and account for a fuller understanding of how that practice fits into a larger ecology of a student's life (or literacy repertoire) raises serious questions about the validity of claims concerning what counts as literacy for these students. We want to highlight a convention we find particularly troubling both methodologically and ethically, that is, the practice of "selective exemplification" in empirical work. Specifically, we refer to zooming in our analytical lens on topics that sensationalize, exoticize, and romanticize, or, conversely, zooming out to essentialize or homogenize English Learners in ways that blind us or our readers to the fuller, more complicated realities of these students' lives. (504)

"framing" English Learners: the way the "problem" of English Learners and other non-dominant students is typically framed. This framework begins with the statistical set-up to the problem that we described above; it is here that a group of non-dominant students are first isolated and identified as a distinct, unified group that is somehow different from an invisible and mostly unspecified norm. In much research, the grouping is based on racial or ethnic-group categorization, not language practices or other distinctions, even when the issue being explored is one of language. (505)

difference framework: white middle-class students are often considered the "norm" in terms home practices, values, beliefs, attitudes, identities, or skills. "The descriptors that researchers use can reveal an underlying set of assumptions about normativity and serve to construct the population as 'other' even when there is no direct comparison with another community or group." (505)

Key ideas:

In this work we have struggled against commonplace approaches to conceptualizing and reporting research that unwittingly create or reinforce deficit views of these students and their communities. (502)

Almost invariably, research reports contextualize the study with statistics about changing demographics in the U.S. as the rationale for the urgency of studying English Learners. Taken together, these data often function to paint pictures of poor, struggling students in schools and communities with limited resources. (502)

We, of course, are not discouraging the use of relevant descriptive statistics as they can serve to identify critical issues and inequities; instead, we call attention to the ways they can constitute deficit-oriented, uncomplicated, and uneven narratives about students for whom English is a second language. (503)

Some time ago, our colleague Elinor Ochs (1979) wrote that researchers engaged in transcribing their participants' talk and interaction are also engaged in the process of building theory about the participants and their practices. Similarly, we argue that the ways we marshal data to make generalizations about what is normative for English Learners, their language and literacy practices, and their home communities build theories about normativity, often without regard to students' existing repertoires of practice or the additional sets of challenges English Learners experience. (504)

There are a number of ways to denote normativity or regularity in English Learners, and one salient tendency in the genre of English Learner studies is to focus on a narrow range of what constitutes the students' literacy toolkit and repertoires. Selectively focusing on what we as researchers find as salient, fascinating, or unusual to us or the field - to the exclusion of the widely diverse range of practices, interests, and proclivities of English Learners- serves to reinforce a kind of analytical reductiveness too often associated with discussions of non-dominant students and communities. (504)

As we try to build a corpus of studies with English Learners, there is a critical need for more nuanced and complete analyses and depictions of students' literacy practices observed across a range of settings, tasks, and contexts over sustained periods of time. (505)

But when the issue is second- language learning, members of this student population should be identified by more than their membership in an ethnic category, and race/ethnicity should not be conflated with language abilities (just as it should not be conflated with social class).(505)

The difference framework has a long history, most perniciously in the overtly deficit-driven notion that some groups of children suffer from "cultural deprivation" or live in "cultures of poverty." The only slightly more benign version of a difference framework - that of "cultural mismatch theory," which spotlights presumed differences between school and home language practices - has had significant longevity in educational research, and, especially, in language and literacy studies. (506)

Heath (1982) - What no bedtime story means

Heath, S. B. (1982). What no bedtime story means: Narrative skills at home and school. Language in Society. 11(2):49-76.

Maintown [is a] white middle-class neighborhoods in a city of the Piedmont Carolina, families of fifteen primary-level school teachers (p.52)

Roadville is a white working-class community of families steeped for four generations in the life of the textile mill. (p57)

Trackton is a working-class black community whose older generations have been brought up on the land, either farming their own land or working for other landowners. However, in the past decade, they have found work in the textile mills. (p57)

Maintown: bedtime stories; children learn/taught what-explanations, classification and knowledge construction, decontextualization, affective comments, reason-explanations (p71)

Roadville: bedtime stories; children learn/taught what-explanations but little or no decontextualization, emphasis on personal experience; no analytic statements or universal truths (p71)

Trackton: no bedtime stories; few occassions for reading to or with children; good storytelling valued; talked about events they witness are rewarded

Learning how to take meaning from writing before one learns to read involves repeated practice in using and learning from language through appropriate participation in literacy events such as exhibitor/questioner and spectator/respondendt yads (Scollon and Scollon 1979) or group negotiation of the meaning of a written text. Children have to learn to select, hold, and retrieve content from books and other written or printed texts in accordance with their community's rules or "ways of taking," and the children's learning follows community paths of language socialization. In each society, certain kinds of childhood participation in literacy events may precedeo thers, as the developmental sequence builds toward the whole complex of home and community behaviors characteristic of the society. [p.70]

Roadville and Trackton tell us that the mainstream type of literacy orientation is not the only type even among Western societies. They also tell us that the mainstream ways of acquiring communicative competence do not offer a universally applicable model of development. They offer proof of Hymes' assertion a decade ago that "it is impossible to generalize validly about 'oral' vs. 'literate' cultures as uniform types" (Hymes 1973: 54). (p.73)

Yet in spite of such warnings and analyses of the uses and functions of writing in the specific proposals for comparative development and organization of cultural systems (cf. Basso 1974: 432), the majority of research on literacy has focused on differences in class, amount of education, and level of civilization among groups having different literacy characteristics. (p.73-74)

"We need, in short, a great deal of ethnography" (Hymes 1973: 57) to provide descriptions of the ways different social groups "take" knowledge from the environment. For written sources, these ways of taking may be analyzed in terms of types of literacy events, such as group negotiation of meaning from written texts, individual "looking things up" in reference books, writing family records in Bibles, and the dozens of other types of occasions when books or other written materials are integral to interpretation in an interaction. These must in turn be analyzed in terms of the specific features of literacy events, such as labelling, what-explanation, affective comments, reason-explanations, and many other possibilities. Literacy events must also be interpreted in relation to the larger sociocultural patterns which they may exemplify or reflect. (p.74)

The culture children learn as they grow up is, in fact, "'ways of taking" meaning from the environment around them. The means of making sense from books and relating their contents to knowledge about the real world is but one "'way of taking" that is often interpreted as "natural" rather than learned. The quote also reminds us that teachers (and researchers alike) have not recognized that ways of taking from books are as much a part of learned behavior as are ways of eating, sitting, playing games, and building houses. (p.49)

In some communities [the] ways of schools and institutions are very similar to the ways learned at home; in other communities the ways of school are merely an overlay on the home-taught ways and may be in conflict with them. (p.50)

Just how does what is frequently termed "the literate tradition" envelope the child in knowledge about interrelationshipsb etween oral and written language, between knowing something and knowing ways of labelling and displaying it? We have even less information about the variety of ways children from non-mainstream homes learn about reading, writing, and using oral language to display knowledge in their preschool environment. The general view has been that whatever it is that mainstream school-oriented homes have, these other homes do not have it; thus these children are not from the literate tradition and are not likely to succeed in school. (p.50)

A key concept for the empirical study of ways of taking meaning from written sources across communities is that of literacy events: occasions in which written language is integral to the nature of participants' interactions and their interpretive processes and strategies. Familiar literacy events for mainstream preschoolers are bedtime stories, reading cereal boxes, stop signs, and television ads, and interpreting instructions for commercial games and toys. In such literacy events, participants follow socially established rules for verbalizing what they know from and about the written material. Each community has rules for socially interacting and sharing knowledge in literacy events. (p.50)

The bedtime story is a major literacy event which helps set patterns of behavior that recur repeatedly through the life of mainstream children and adults. (p.51)

Before the age of two, the child is socialized into the "'initiation-reply-evaluation sequences" repeatedly described as the central structural feature of classroom lessons. (p.51) Training in ways of responding to this pattern begins very early in the labelling activities of mainstream parents and children. (p.52)

Reading for comprehension involves an internal replaying of the same types of questions adults ask children of bedtime stories. We seek what-explanations, asking what the topic is, establishing it as predictable and recognizing it in new situational contexts by classifying and categorizing it in our mind with other phenomena. (p54)

These various ways of taking [from books] are sometimes referred to as "cognitive styles" or "learning styles." It is generally accepted in the research literature that they are influenced by early socialization experiences and correlated with such features of the society in which the child is reared as social organization, reliance on authority, male-female roles, and so on. These styles are often seen as two contrasting types, most frequently termed "field independent-field dependent" (Witkin et al. 1966) or "analytic-relational" (Kagan, Sigel, and Moss I963; Cohen 1968, 1969, 1971). The analytic field-independent style is generally presented as that which correlates positively with high achievement and general academic and social success in school. Several studies discuss ways in which this style is played out in school - in preferred ways of responding to pictures and written text and selecting from among a choice of answers to test items. (p55)

In both [the Roadville and Trackton] communities, children go to school with certain expectancies of print and, in Trackton especially, children have a keen sense that reading is something one does to learn something one needs to know (Heath 1980). ... Roadville and Trackton view children's learning of language from two radically different perspectives: in Trackton, children "learn to talk," in Roadville, adults "teach them how to talk." (p57)

Nieto, S. (2005). Public education in the twentieth century and beyond

Nieto, S. (2005). Public education in the twentieth century and beyond: High hopes, broken promises, and an uncertain future. Harvard Educational Review 75(1):43-64.

Theories used explain the underachievement of students of diverse cultural, linguistic, and racial backgrounds.

1. Genetic and Cultural Inferiority - proponents assert that students of racial minority and economically poor backgrounds are genetically or culturally inferior

2. Economic and Social Reproduction Theories - schools tend to serve the interests of the dominant classes by reproducing the economic and social relations of society; schools help to create and maintain these inequalities.

3. Cultural Incompatibility Theory - school culture and home culture are often at odds, and the result is a “cultural clash” that gets in the way of student learning

4. Sociocultural Explanations for School Achievement - cultural practices of particular communities are linked with their students' learning in school settings. Shirley Brice Heath's (1983) classic research with a Black community that she called "Trackton" is a persuasive example of the power of aligning teaching to students' cultural practices.

5. Students as Castelike Minorities - According to Ogbu, given the long history of discrimination and racism in the schools, involuntary minority children and their families are often distrustful of the education system. It is not unusual for students from these groups to engage in what Ogbu called cultural inversion, that is, to resist acquiring and demonstrating the culture and cognitive styles identified with the dominant group.

6. Resistance theory, as articulated by scholars such as Henry Giroux (1983), Jim Cummins (1996), Herb Kohl (1994), and others, adds another layer to the explanation of school failure. According to this theory, not learning what schools teach can be interpreted as a form of political resistance.

7. Care, Student Achievement, and Social Capital - for Nel Noddings (1992), care is just as — and in some cases, even more — important than entrenched structural conditions that influence student learning. Valenzuela (1999): subtractive schooling concept; Ricardo Stanton-Salazar (1997): social capital networks framework

Newer theories: race, context of incorporation, and others - are also at work. Haitians, Nicaraguans, and Mexicans as economic refugees, who have significantly lower earnings than Cubans and Vietnamese, even after controlling for level of education, knowledge of English, and occupation. Alejandro Portes and Rubén Rumbaut (2001) found that immigrants fleeing from Communism are received more favorably than those fleeing economic exploitation.

Response to an OpEd by Nicholas Kristof

Below is my response to Response to an OpEd piece "How Mrs. Grady Transformed Olly Neal" by Nicholas Kristof (Published: January 21, 2012 in The New York Times)

I think we should celebrate teachers like Mildred Grady who made an extra effort to help a child and encourage his interest in reading, and Olly Neal who succeeded under challenging circumstances. Both sound like exceptional human beings.

However, I think Kristof's column and the simple narrative that a single great teacher transformed the life of an "at-risk" youth and was the key factor that enabled him to become educated, get a law degree and eventually become a court judge, is somewhat misleading. The column obscures the other positive influences in Olly Neal's life that were probably as or more important than Mrs. Grady - parents, family, mentors, etc. In addition, it is unlikely that Mr. Neal would have succeeded if he didn't have many competent and caring teachers through his life - before and after Mrs. Grady - and access to functional education systems.

While a great teacher can make a difference with some troubled, surly kids in a high-poverty environment, the odds of this happening are very low struggling in many inner-city schools today. The schools with the most troubled kids in high-poverty environments tend to have the toughest working conditions, worst funding, and weakest leadership that few good teachers would want to work or stay at.

What is Culture?

Excerpt from Michael Cole's Cultural Psychology (1996):
Culture, according to Hutchins, should be thought of as a process, not as "any collection of things, whether tangible or abstract." Culture "is a process and the 'things' that appear on list-like definitions of culture are residua of the process. Culture is an adaptive process that accumulates the partial solutions to frequently encountered problems." (Hutchins, p. 354)

---

Culture includes: artifacts, schemas, scripts, models, practices, heritages, history, activity systems

---

Excerpt from McDermott & Varenne (1995). Culture as Disability:

Anthropologists define culture as well-bound containers of coherence that mark off different kinds of people living in their various ways, each kind separated from the others by a particular version of coherence, a particular way of making sense and meaning. (p. 325)

The coherence of any culture is not given by members being the same, nor by members knowing the same things. Instead, the coherence of a culture is crafted from the partial and mutually dependent knowledge of each person caught in the process and depends, in the long run, on the work they do together. Life in culture, Bakhtin (1984[1940]) reminds us, is polyphonous and multivocalic; it is made of the voices of many, each one brought to life and made significant by the others, only sometimes by being the same, more often by being different, more dramatically by being contradictory. Culture is not so much a product of sharing as a product of people hammering each other into shape with the well-structured tools already available. We need to think of culture as this very process of hammering a world. When anthropologists instinctively celebrate the coherence of culture, they imply that all the people in the culture are the same, as if stereotyping is a worthy practice as long as it is done by professionals. Thick brush-stroke accounts of Samoans or Balinese, to stay with Margaret Mead, may give some hints as to what Samoans and Balinese must deal with in their daily life, but they can greatly distort the complexity of Samoans and Balinese as people. The coherence of culture is something many individuals, in multiple realities, manage to achieve together; it is never simply the property of individual persons. (p.326)

The anthropological instinct has been perhaps most destructive when applied to the divisions and inequalities that exist inside a presumed cultural container, that is, the culture “of which they are a member,” “to which they belong,” or “in which they participate.” The problem in assuming that there is one way to be in a culture encourages the misunderstanding that those who are different from perceived norms are missing something, that it is their doing, that they are locked out for a reason, that they are in fact, in reality, disabled. If it is distorting to describe Samoans and Balinese without an account of the full range of diversity to be found in Samoa or Bali, imagine how distorting it can be in complex divided fields like the United States.

When culture is understood as the knowledge that people need for living with each other, it is easy to focus on how some always appear to have more cultural knowledge than others, that some can be a part of everything and others not, that some are able and others not. Before entering the Country of the Blind, Nunez thought that sight was essential to being fully cultured and that having sight in a world of people who cannot see would net him the cultural capital of a king. The anthropological instinct teaches us that he was arrogant to think he knew better and foolish to not learn from his masterful subjects. The instinct gives us an essential insight, and we can be thankful that anthropology has taken its place in the human sciences.