Rick Warner, Associate Professor of History, Jane and Frederick Hadley Chair in History, Wabash College
Contact at: warnerri@wabash.edu
Over the past generation or so, History teachers (and I suspect others) have moved from the older model of assessing students through multiple-choice (MC) examinations, which are often thought to be old-fashioned or perhaps charmingly antiquated methods of measuring student outcomes.
There are multiple reasons for this trend away from the MC exam. From the standpoint of historical scholarship, our field was influenced to fair degree by postmodern critiques of the “objective historical fact.”[1] Rather than understanding our work as an exercise in mining provable historical nuggets, the emphasis has shifted to recognizing the importance of perspective, both on the part of historical actors (through primary sources) and by those of us who narrate history through oral and written texts (secondary sources). Narratives about the past are, in a sense, “plotted” either consciously or not.[2]
I like to illustrate this phenomenon by asking students why we fought the Iraq wars. Some will say, “for oil and money,” others will say, “to fight Muslim extremism.” There are other possibilities. But, in one hundred years, how will a historian answer this question? The point is that rather than singular facts, most interesting historical questions require contextualization and analysis of perspectives and surrounding historical information from the time. The multiple exam choices of yesteryear cannot engage such questions as well as essay narratives, so historians have moved toward written narratives for student evaluation.
Meanwhile, my discipline, among others, has become more intentional about the definition of pedagogical motives and goals. In the past, multiple-choice questions measured the ability for students to regurgitate narrow slivers of information, a process which some of us call “factoid recall.” While there may be some trivial interest in whether George Washington had wooden teeth, or if Grant is actually buried in Grant’s tomb, we prefer nowadays to believe that we are equipping students with historical thinking skills, to be added to a quiver of other critical thinking abilities developed through liberal arts education.
ADVANCED PLACEMENT HISTORY CHANGES
Much of my thinking about these matters has developed from twenty years of working with College Board, specifically on their World History Advanced Placement exam. In recent years all three AP History exams – U.S., Europe, and World – have cooperated to identify a common set of “History Reasoning Skills” that can be applied to Learning Objectives through the three exams.[3] In its latest iteration, the curriculum identifies four reasoning skills. All exam questions, both multiple choice and essay, are coded to one or more of these four skills:
- Contextualization
- Comparison
- Causation
- Continuity and Change Over Time Interactions
Questions are additionally coded to “Thematic Learning Objectives”:
- Interaction Between Humans and the Environment (ENV)
- Development and Interaction of Cultures (CUL)
- State Building, Expansion, and Conflict (SB)
- Creation, Expansion, and Interaction of Economic Systems (ECON)
- Development and Transformation of Social Structures (SOC)
More complete explanations of these Skills and Learning Objectives can be found in the Curriculum Framework documents.
MULTIPLE CHOICE EXAMPLES FROM WORLD HISTORY AP
In the new AP framework, three questions are typically grouped together in a cluster, as they respond to a “stimulus,” sometimes called an “asset.” These can be a primary or secondary passage, an image, or another type of visual such as a graph.
Here is an example of a question set drawn from the Course and Exam Description (CED) that can be found on AP Central:
Questions 7–9 refer to the passage below.
The ruler is a boat; people are the water. The water can carry the boat; the water can capsize the boat. . . . A man may be the descendant of kings, lords, or nobles, but if he does not observe the norms of ritual and proper behavior he must be relegated to the status of a commoner. Similarly, he may be a descendant of commoners, but if he accumulates learning of the texts, corrects his behavior, and observes the norms of ritual and proper behavior—then he must be elevated to the ranks of high ministers, lords, and nobles.
Xunzi, Chinese philosopher, circa 250 BCE.
7. According to the passage, Xunzi was advocating an approach to governance that most clearly reflected the principles of:
A. Daoism
B. Legalism
C. Confucianism
D. Buddhism
8. Xunzi’s idealized vision of Chinese society in the passage differs most strongly from the social structure of which of the following:
A. Roman society during the late empire
B. Hindu society in South Asia during the Gupta Empire
C. Muslim society during the early Caliphates
D. Mongol society during the period of Mongol conquests
9. Ideas similar to those expressed in the passage have directly contributed to the development of which of the following aspects of later Chinese imperial history?
A. The long-standing tradition of Chinese leadership in technological, agricultural, and commercial innovation relative to the rest of the world;
B. The expectation that emperors must be judged by a different set of ethical standards all other members of society;
C. The virtual elimination of the threat of rebellion against established political authority;
D. The practice of recruiting capable bureaucrats on the basis of educational achievement rather than noble birth
[4]
DESCRIPTION OF HISTORICAL REASONING SKILLS (HRS) AND LEARNING OBJECTIVES (LOS):
Question 7: Comparison, CUL, SOC, answer is “C.” The student is asked to compare the four major belief systems. Both cultural and social structural aspects are involved in the question.
Question 8: Comparison, SOC, answer is “B.” The student is asked to compare major societies that are studied during the course, specifically the social structure of each.
Question 9: Causation, SB, SOC, answer is “D.” The student is asked to imagine ways that classical Chinese thought influenced later social structure and state building, in particular the philosophical underpinnings of Chinese bureaucracy in future dynasties (including the role of neo-Confucianism and civil service examination system, for example).
Not all questions are equally difficult. In test development lingo, these questions can “discriminate” well. From what I can see here, I would predict that fewer students will answer question 8 correctly than 7, and fewer still will answer question 9 correctly. Another way to understand this is that questions are “scaffolded” to move up the “Bloom’s Taxonomy” levels of analysis. As a result, multiple choice tests produce a wider array of results than essay questions in most classrooms. The eventual outcome can of course by massaged by “curving” the scores, if that seems appropriate.
OTHER APPLICATIONS
In my own practice, while I still assign numerous essays to History students, I have been supplementing class assessments with this new genre of questions. In general, I write stand-alone questions that I believe rely on skill development as much as information recall. Here are a few that I recently used:
Which of the following societies is this art work most likely from?
(Hagia Sophia, Istanbul, Turkey)
A. Shang Chinese
B. Byzantium
C. Mongol
D. Safavid
Answer: B. The student will have seen Byzantine art, and should understand something about the religious iconography of that society.
How does the picture below reflect a Buddhist worldview?
(Borobudur Temple, Magelang, Central Java, Indonesia)
A. Situated on a particular meridian, the sun god is honored.
B. The cost of constructing the monument reveals materialist motives
C. The circular space on top represents the key Buddhist gods.
D. The various levels represent states in the pursuit of enlightenment.
Answer is D. The student needs to know key concepts about the Buddhist faith (or what it is not) and apply these to the image.
The following graph is based on information drawn from records in England for the years noted:
(https://www.economist.com/blogs/freeexchange/2013/10/economic-history-1)
What conclusion can be drawn from the above graph?
A. The cost of building materials went up from 1450 to 1500 in Europe.
B. The voyages of Columbus benefited carpenters due to extra ship building.
C. The Black Death benefited peasants who survived it.
D. Rulers were able to tighten their grip on European society following the Black Death.
The answer is C. The student would need to know the approximate (not exact!) dates of the Black Death, and to reason the economic reasons that a smaller labor pool would prove advantageous for the peasant class.
CONCLUSION
These multiple choice questions will not replace narrative essays in History learning and assessment. The College Board recognizes this, and at least half of the final score (1-5) on the exams will be based on essays that are evaluated by a corps of Readers each summer. These questions have benefited my own practice as a History teacher over the past few years. I have come to believe that students learn in different ways: through visuals, reading, experience, and so forth. Some students – notably science majors – have improved their performance in my classes with the addition of the multiple choice tests. Through my work with College Board, I believe that these assessment devices can be constructed with an eye toward pedagogical intentionality, and that through proper application of rubrics even the essays can be scientifically evaluated.
And as for grading multiple choice exams, well, this can be done while sitting through a single episode of “Downton Abbey”!
[1] Peter Novick, That Noble Dream: The “Objectivity Question” and the American Historical Profession, New York: Cambridge UP, 1988.
[2] Initial narrative analysis along these lines was most famously published by Hayden White in his Metahistory: The Historical Imagination in Nineteenth-Century Europe, Baltimore: Johns Hopkins Press, 1973.
[3] Further information, including copies of the AP Course and Exam Descriptions, can be found at AP Central.
[4] World History CED, 186. Accessed 8 December 2017.
