Cognitive Science and Assessment
ERIC Identifier: ED481716
Publication Date: 2003-06-00
Author: Boston, Carol
Source: ERIC Clearinghouse on Assessment and Evaluation College Park MD.
Cognitive science is devoted to the study of how people think and learn and how, when,
and whether they use what they know to solve problems (Greeno, Collins, & Resnick,
1997; National Research Council, 2001). The cognitive perspective in education
encompasses how learners develop and structure their knowledge in specific subject
areas and how assessment tasks might be designed to enable students to demonstrate
the knowledge and cognitive processes necessary to be judged proficient in these
subject areas. This Digest provides educators with an overview of some important
facets of cognitive science research and suggests implications for classroom
HOW DO EXPERTS AND NOVICES DIFFER IN THEIR APPROACH TO PROBLEMS?
Education researchers study the thinking of experts in various subject areas to gain an
understanding of what concepts and procedures are most important to teach and how
they are interrelated. The concept is that educators can and should be moving students
along a continuum toward real-world subject mastery based on a deep understanding of
how subject knowledge is organized (Bereiter & Scardamalia, 1986).
When faced with a problem, learners tend to search their memories for a schema, or
learned technique for organizing and interpreting information in a certain subject, in
order to solve it (Rumelhart, 1980). Over time, individuals build mental models to guide
their problem solving efficiently so they do not depend on trial-and-error approaches
and can instead create analogies and make inferences to support new learning (Glaser
& Baxter, 1999).
When compared with novice learners, experts in a subject are notable for how
well-organized their knowledge is, which in turn enables them to see patterns quickly,
recall information, and study novel problems in light of concepts and principles they
know already (Glaser & Chi, 1988). In other words, their schemas are well-connected
and they are able to retrieve chunks of information relevant to a task at hand. Experts
also have strong problem-solving skills. They know what they know and what they don't
know, and plan and monitor the implementation of various mental strategies (Hatano,
COGNITIVE SCIENCE IN THE CLASSROOM
Ideally, developmental models of learning could be created that note the typical
progression and milestones as a learner advances from novice to competent to expert
and describe the types of experiences that lead to change. For example, students
generally have naive or intuitive understandings of the sciences, based in part on
misconceptions that are corrected as they are exposed to new learning (e.g., Gabel,
1994, Feldman & Minstrell, 2000). And while there are individual differences among
learners, when large samples are studied, patterns tend to emerge, particularly related
to erroneous beliefs and incorrect procedures. For example, there appear to be a
certain limited number of "subtraction bugs" that account for almost all of the ways
young children make mistakes when learning to subtract two- or three-digit numbers,
and these are constant even across languages (Brown and Burton, 1978).
Allowing for variations among learners, it is possible to discover the most common
pathways toward acquiring knowledge and use this information diagnostically. For
example, Case, Griffin, and colleagues have developed an assessment tool based on
their empirical research regarding how children from ages 4 to 10 change in their
conception of numbers through growth and practice.
More research has been done about domain structure in some disciplines than in
others. Mathematics, physics, beginning reading, and U.S. history are among the areas
that have been studied (see, for example, Niemi, 1996, and Wineburg, 1996).
Subject-area standards such as the National Council of Teachers of Mathematics
Standards generally reflect current thinking on cognitive processes and are a good
place for teachers to begin their explorations of this topic. The National Research
Council's How People Learn: Brain, Mind, Experience, and School
(http://stills.nap.edu/html/howpeople1/) provides another helpful introduction.
- While 4-year-olds can count
groups of objects, they have to guess if they face a theoretical question such as, "Which
is more--four or five?"
- Between 4 and 6, most children develop a "mental number line"
that helps them envision the answer to such a question, even when actual objects aren't
- Between 6 and 8, children gradually come to envision other number lines for
counting by 2s, 5s, 10s, and 100s.
- By 10, many children have a better understanding of
the base-10 number system, which enables them to reach a more sophisticated
understanding of concepts such as regrouping and estimation (Case, 1996; Griffin and
Case, 1997). Teachers can use assessments based on this research to determine their
next steps in arithmetic instruction.
HOW DO LEARNERS STORE AND ACCESS KNOWLEDGE?
Memory may be divided into two types: short-term, or working memory, which
determines how much mental processing can go on at any one time, and long-term
memory, where people organize their content knowledge. Short-term memory, or
working memory, is connected with fluid intelligence, or the ability to solve new and
unusual problems, while long-term memory is connected to crystallized intelligence, or
the bringing of past experience to bear on current problems (Anderson, Greeno, Reder,
and Simon, 2000). When students are learning a new skill, they must rely heavily on
their working memory to represent the task and may need to talk themselves through a
task. As the skill moves into long-term memory, it becomes fluent, and eventually,
automatic (Anderson, 1982).
To support the learning process, students can be taught meta-cognitive skills, or
techniques to reflect on and assess their own thinking. To improve reading
comprehension, for example, young children can be taught to monitor their
understanding of passages by asking questions, summarizing, clarifying any
uncertainties, and predicting next events (Palincsar & Brown, 1984).
HOW CAN ASSESSMENT DESIGNERS USE FINDINGS FROM COGNITIVE SCIENCE?
The design of any assessment should begin with a statement of purpose for the
assessment and a definition of the particular subject area or content domain. How do
people demonstrate knowledge and become competent in this domain? What important
aspects of learning do we want to draw inferences from when measuring student
achievement in a given subject area? What situations and tasks can we observe to
make the appropriate inferences?
Teachers should also be aware that acquiring important knowledge and skills at an
in-depth level takes a significant amount of time, practice, and feedback.
- Cognitive science calls for test developers to:
- Work from a deep knowledge of the
central concepts and principles of a given subject area, and the most important related
- Identify or develop those tasks that allow students to demonstrate their
understanding and skills in these areas, as opposed to rote memorization.
- Make sure
tasks or questions are sufficiently complex to get at how students have organized their
knowledge and how and when they use it.
- Emphasize the contents of long-term
memory rather than short-term, or working, memory by not burdening test-takers with
requirements to track a large number of response options or major quantities of
extraneous information while answering a question.
- Emphasize relevant
constructs--for example, a mathematics assessment should not over-emphasize
reading and writing, unless communicating about mathematics is the skill to be
- Not limit choice of item format. Both multiple-choice and
performance-based assessments have the potential to be effective or ineffective.
Carefully constructed multiple-choice questions can tap complex cognitive processes,
not just lower level skills, as traditionally believed. And performance assessments,
though generally praised for capturing higher level skills, may inadvertently focus on
lower level skills (Baxter & Glaser, 1998; Hamilton, Nussbaum, and Snow, 1997; Linn,
Baker, & Dunbar, 1991).
- Regard task difficulty in terms of underlying knowledge of
cognitive processes required, rather than statistical information such as how many
respondents answered correctly.
- At the classroom assessment level, cognitive science findings encourage teachers to:
- Teach learners how and when to apply various approaches and procedures.
- Teach meta-cognitive skills within content areas so learners become capable of directing their
thinking and reflecting on their progress.
- Observe students as they solve problems.
- Have students think aloud as they work or describe the reasoning that leads them to a
- Analyze student errors on assignments or tests to determine which
students got a question or problem wrong and why it appeared difficult for them.
Knowing the source of difficulty can lead to more targeted, effective remediation.
This publication was prepared with funding from the Office of Educational Research and
Improvement, U.S. Department of Education, under contract ED99CO0032. The
opinions expressed in this report do not necessarily reflect the positions of policies of
OERI or the U.S. Department of Education. Permission is granted to copy and distribute
this ERIC/AE Digest.
Anderson, J. (1982). Acquisition of cognitive skill. Psychological Review, 89, 369-406.
Anderson, J., Greeno, J., Reder, L., and Simon, H.A. (2000). Perspectives on learning,
thinking, and activity. Educational Researcher, 229 (4): 11-13.
Baxter, G. and Glaser, R. (1998). Investigating the cognitive complexity of science
assessments. Educational Measurement: Issues and Practices, 17 (3): 37-45.
Bereiter, C. & Scardamalia, M.(1986). Educational relevance in the study of expertise.
Interchange, 17 (2): 10-19.
Brown, J.S. and Burton, R.R. (1978). Diagnostic models for procedural bugs in basic
mathematical skills. Cognitive Science, 2, 155-192.
Case, R. (1996). Introduction - Reconceptualizing the development of children's
conceptual structures and their development in middle childhood. Monographs of the
Society for Research in Child Development, 61 (1-2): 1-26.
Feldman, A., & Minstrell, J. (2000). Action research as a research methodology for the
study of the teaching and learning of science. In E. Kelly & R. Lesh (Eds.), Handbook of Research Design in Mathematics and Science Education. Mahwah, NJ: Erlbaum.
Gabel, D., ed. (1994). Handbook of Research on Science Teaching and Learning. New York: Macmillan.
Glaser, R. and Baxter, G. (1999). Assessing active knowledge. Paper presented at the
1999 CRESST Conference, Benchmarks for Accountability: Are We There Yet? UCLA,
Glaser, R. and Chi, M. (1988). Overview in M. Chi, R. Glaser, & M. Farr (Eds.), The
Nature of Expertise (pp. xv-xxvii). Hillsdale, NJ: Erlbaum.
Greeno, J.G., Collins, A.M., & Resnick, L.B. (1997). Cognition and learning. In D.
Berliner & R. Calfee (Eds.), Handbook of Educational Psychology (pp. 15-47). New
York: Simon & Schuster Macmillan.
Griffin, S., and Case, R. (1997). Re-thinking the primary school math curriculum: An
approach based on cognitive science. Issues in Education, 3, 1-65.
Hamilton, L., Nussbaum, E., & Snow, R. (1997). Interview procedures for validating
science assessments. Applied Measurement in Education, 10, 181-200.
Hatano, G. (1990). The nature of everyday science: A brief introduction. British Journal
of Developmental Psychology, 8, 245-250.
Linn, R., Baker, E., & Dunbar, S. (1991). Complex, performance-based assessment:
Expectations and validation criteria. Educational Researcher, 20 (8):15-21.
National Research Council (2001). Knowing What Students Know: The Science and Design of Educational Assessment. Washington, DC: National Academy Press.
Niemi, D. (1996). Assessing conceptual understanding in mathematics:
Representations, problem solutions, justifications, and explanations. Journal of
Educational Research, 89, 351-363.
Palinscar, A. and Brown, A. (1984). Reciprocal teaching of comprehension-fostering
and comprehension-monitoring activities. Cognition and Instruction, 1, 117-175.
Rumelhart, D. A. (1980). Schemata: The building blocks of cognition. In R. Spiro, B.
Bruce, & W. Brewer (Eds). Theoretical Issues in Reading Comprehension (pp. 33-58). Hillsdale, NJ: Erlbaum.
Wineburg, S. S. (1996). The psychology of learning and teaching history. In D. Berliner
& R. Calfee (Eds.), Handbook of Educational Psychology (pp. 423-437). New York: Simon & Schuster Macmillan.
Promoting Critical Thinking |
Menu Page |
Parenting the Next Generation |
de Bono's Thinking Structure