They combine errors of making dubious causal claims and misusing rankings, and by ignoring evidence from low or average scoring nations on the same policy question, produces a triple error, a true whopper in misinterpreting international test scores, claims Tom Loveless.
“Conclusion
International test scores are a valuable resource for policy makers and of great interest to the public. As this section of the Brown Center Report has illustrated, the results are also vulnerable to misinterpretation, especially when cited as evidence in political battles over the wisdom of adopting particular policies. Three misinterpretations are common. First, dubious claims of causality. Arguments have been made that Poland’s tracking reforms spurred achievement gains on the PISA reading test from 2000–2009. It is plausible that tracking reform contributed to Poland’s success, but the evidence is weak. Other countries accomplished gains just as large as Poland’s without engaging in tracking reform. Many of them also boosted the scores of low achievers as much as Poland did. Moreover, Poland adopted several important reforms at the same time that tracking reform took place, and it is impossible to disentangle the effects of one reform from the others. Polish attitudes towards education shifted dramatically during this period and may have provided cultural support for achievement. A second common mistake is the misuse of national rankings. The test scores underlying two adjacent rankings, or even several close rankings, may not be statistically significantly different. Rankings are not equal interval—they differ in various parts of the distribution—so a nation may jump several rankings with a gain that is actually smaller than that of a country whose ranking stays the same. Rankings must be interpreted with great caution.Finally, the A+ country fallacy is a common mistake. Pointing to a single, highscoring country, or a group of them, and declaring that one or more of their policies should be adopted by other countries is misguided. It combines the errors of making dubious causal claims and misusing rankings, and by ignoring evidence from low or average scoring nations on the same policy question, produces a triple error, a true whopper in misinterpreting international test scores.”
[Source: The Brown Center Report on American Education, Page 31]
Misinterpreting international test scores
The 2012 Brown Center Report on American Education:
Today there are more than 1,400 billionaires and 93,000 “ultra-high-net-worth” individuals. Many of these are young, entrepreneurs who have made their money in technology and are interested in using that wealth to slay some of the world’s grand challenges. We call these individuals Technophilanthropists.
Chapter 1: Our Grandest Challenge
Our Grandest Challenge – We open with the history of aluminum and the idea of technology as a resource-liberating mechanism, freeing us from the threat of scarcity. Visit Masdar, Abu Dhabi’s energy city of the future. Learn about the Limits to Growth, a pessimistic vision of the future which set expectations for many generations, and drove efforts to control population growth. The Chapter closes outlining the possibility of Abundance and outlining the four forces conspiring to transform our planet.
Your education doesn’t have to stop once you get out of school—being free of the classroom just means you have more control over what you learn and when you learn it. We’ve put together a curriculum of some of the best free online classes available on the web this spring for the first term of Lifehacker U, our regularly-updating guide to improving your life with free, online college-level classes. Let’s get started.
Title photo remixed from an original by Stuart Seeger.
The Institute for Public Knowledge (IPK) brings theoretically serious scholarship to bear on major public issues. Located at NYU, it nurtures collaboration among social researchers in New York and around the world. It builds bridges between university-based researchers and organizations pursuing practical action. It supports communication between researchers and broader publics. And it examines transformations in the public sphere, social science, and the university as a social institution as these change the conditions for public knowledge.
Roundtable held at the Institute for Public Knowledge
IPK Director
• Craig Calhoun | Director
• University Professor of Social Science | New York University
IPK Staff
• Samuel Carter | Assistant Director
• Assistant Director | Institute for Public Knowledge
• Jessica Coffey | Administrator
•
IPK Senior Fellows
• Arjun Appadurai
• Goddard Professor of Media, Culture, and Communication | New York University
• Richard Burdett
• IPK POIESIS Senior Fellow | Director, Urban Age Programme | The London School of Economics and Political Science
• Faisal Devji
• Reader in Modern South Asian History | Oxford University
• Eric Klinenberg
• Professor of Sociology at NYU | Editor of Public Culture | New York University
• Michael Likosky
• Director | Center on Law and Public Finance
• David Ludden
• Professor of History | NYU
• Peter Alexander Meyers
• Professor of American Studies | Université Paris III – Sorbonne Nouvelle
• Harvey Molotch
• Professor of Social and Cultural Analysis, Sociology | NYU
• Richard Sennett
• University Professor, NYU | Professor of Sociology, London School of Economics
• Judith Stacey
• Professor | Social and Cultural Analysis and Sociology
• Charles Taylor
• Professor Emeritus | McGill
• Peter van der Veer
• Professor | Max Planck Institute for the Study of Religious and Ethnic Diversity
Poiesis Fellows
• Ash Amin
• Professor of Geography | Durham University
• Nerea Calvillo
• Principal | C+arquitectos
• Peter Claussen
• Academic Consultant and Eco-Farmer | Former Director BMW Plant Leipzig
• Manthia Diawara
• University Professor | New York University
• Naresh Fernandes
• Editor in Chief | Time Out India
• Evelyn Fox Keller
• Professor, History and Philosophy of Science | Massachusetts Institute of Technology
• Gerald Frug
• Louis D. Brandeis Professor of Law | Harvard University
• Nilufer Gole
• Professor of Sociology | Ecole des Hautes Etudes
• Orit Halpern
• Assistant Professor, Committee for Historical Studies | The New School for Social Research
• Haiyan Huang
• Professor, School of Art and Design | Xi’an University of Technology
• Lou Kauffman
• Professor, Department of Mathematics, Statistics, and Computer Science | University of Illinois – Chicago
• Monika Krause
• Lecturer in Sociology | University of Kent
• Jesse LeCavalier
• Doctoral Candidate, Retail Logistics and Urbanism | Swiss Federal Institute of Technology, Zurich
• Klaus Mainzer
• Director of the Carl von Linde-Academy | Technical University of Munich
• Clapperton Mavhunga
• Assistant Professor, Program in Science, Technology, and Society | Massachusetts Institute of Technology
• Michael McQuarrie
• Assistant Professor, Department of Sociology | University of California, Davis
• Birgit Meyer
• Professor, Department of Social and Cultural Anthropology | VU University Amsterdam
• Wolfgang Pietsch
• Research Assistant in Philosophy of Science | Technische Universität München, Germany
• Saskia Sassen
• Robert S. Lynd Professor of Sociology | Columbia University
• Harel Shapira
• Postdoctoral Fellow | Institute for Public Knowledge
• Cassim Shepard
• Director, Urban Omnibus | The Architectural League of New York
• Alejandro Zaera-Polo
• Founding Partner | Foreign Office Architects
IPK Scholars
• Hillary Angelo
• PhD Candidate in Sociology | New York University
• Natalia Besedovsky
• PhD Candidate in Sociology | Humboldt Universität
• Ruth Braunstein
• PhD Candidate in Sociology | New York University
• Nandi Dill
• PhD Candidate in Sociology | New York University
• Simon Head
• Fellow | Rothermere American Institute, Oxford
• Kaisa Ketokivi
• Post-Doctoral Researcher, Lecturer | University of Helsinki, Finland
• Andreas Koller
• Research Fellow | Social Science Research Council
• Pierluigi Musarò
• Assistant Professor, Faculty of Political Science | Bologna University
• Laura Norén
• PhD Candidate in Sociology | New York University
• Martha Poon
• Visiting Scholar | University of California, San Diego
• Besnik Pula
• PhD Candidate in Sociology | University of Michigan
• Sasha Roseneil
• Professor of Sociology and Social Theory and Director of the Birkbeck Institute for Social Research | University of London
• Jennifer Telesca
• PhD Candidate | Media, Culture & Communication | | New York University
• Jonathan VanAntwerpen
• PhD Candidate in Sociology | University of California Berkeley
• Robert Wosnitzer
• PhD Candidate in Media, Culture, and Communication | New York University
IPK Alumni
• Esther Hio-Tong Castillo
• Masters Candidate in the Draper Interdisciplinary Program | New York University
• Ingrid Erickson
• Research Fellow/Program Officer | Social Science Research Council
• Nicolas Guilhot
• Program Officer | Social Science Research Council
• Hannah Jones
• Doctoral Student | Goldsmiths, University of London
• Joel Kahn
• Emeritus Professor, Sociology & Anthropology Program | La Trobe University
• Øjvind Larsen
• Professor of Philosophy and Sociology | Copenhagen Business School
• Ted Magder
• Associate Professor of Media, Culture, and Communication | NYU
• Manjari Mahajan
• Program Officer | Social Science Research Council
• Tey Meadow
• Doctoral Candidate | Department of Sociology, New York University
• George J.A. Murray
• Visiting Assistant Professor | Davidson College
• Willem Schinkel
• Associate Professor of Theoretical Sociology | Erasmus University Rotterdam, the Netherlands
• Amrita Shah
• Journalist and Nonfiction Writer
• Matthew Noah Smith
• Assistant Professor of Philosophy | Yale University
• Raluca Soreanu
• Research Associate | SOAS, University of London, UK
• Siovahn Walker
• Program Officer | Social Science Research Council
• Xuan Zuo
• Graduate Student | Beijing Foreign Studies University
“Sir Ken Robinson, PhD is an internationally recognized leader in the development of education, creativity and innovation. He is also one of the world’s leading speakers with a profound impact on audiences everywhere. The videos of his famous 2006 and 2010 talks to the prestigious TED Conference have been seen by an estimated 200 million people in over 150 countries.”
This animate was adapted from a talk given at the RSA by Sir Ken Robinson, world-renowned education and creativity expert and recipient of the RSA’s Benjamin Franklin award.
Your education doesn’t have to stop once you get out of school—being free of the classroom just means you have more control over what you learn and when you learn it. We’ve put together a curriculum of some of the best free online classes available on the web this spring for the first term of Lifehacker U, our regularly-updating guide to improving your life with free, online college-level classes. Let’s get started. 
