You can’t IMPROVE what you can’t MEASURE

By Grace Shangkuan Koo
Philippine Daily Inquirer

 21share161 132

Dr. Andreas Schleicher is a man with a mission. He holds thousands of pieces of data on learning outcomes from 74 countries.

He goes around the world making these statistics transparent to government officials, education ministers, university presidents, school superintendents, teachers, parents, and even bankers, hoping they would appreciate the implications of these numbers.

At the recent 37th Annual Conference of the International Association for Educational Assessment, which was held in Manila and hosted by the Center for Educational Measurement, he delivered the closing address with unmistakable urgency.

His message: Systems that are honest to themselves will understand that competencies that are measured have a very powerful impact on one’s life chances.

World’s schoolmaster

Originally from Hamburg and a physicist by training, Schleicher, dubbed the world’s schoolmaster, is one of the most influential education experts that you have not heard of.

When he first joined the Organization for Economic Cooperation and Development (OECD) in 1996, he and his colleagues proposed to design a test that would shift from measuring inputs (like spending on schools) to outputs (how much kids are learning).

The result was the Program for International Student Assessment, globally known as PISA, a two-hour test that assesses students’ capacity to extrapolate and apply creatively from what they know.

PISA tests are opposite to common school tests. Schleicher pointed out that test items that were easiest to assess were the easiest to teach, but also the easiest to lose.

What are some test items in PISA? In reading, for instance, students are given two texts about graffiti—one negatively as vandalism, another positively as art. Students are then asked which side they choose, and to judge the rationale of their arguments. In mathematics, students who are good in equations and formulas may not do well in PISA. The test requires students to use math in understanding the structure in the real world and to decide which tools are relevant and how to apply them in problem solving.

In 2000, 32 countries of OECD signed in to participate in PISA. By 2011, even non-OECD members have joined the assessment, making a total of 74 participating countries. By the way, the Philippines is not a PISA participant.

With statistics culled from 28 million 15-year-old students, Schleicher has revealed unexpected, if not shocking, results over the years.

For one, the United States has dropped from rank No. 1 to 26 in the 2011 results. The current rankings show that many of the highest-performing school systems are in Asia—specifically, China, Korea, Hong Kong, Singapore and Japan.

Not how much but where

The results further showed that outcome is not about how much a country is spending on education, but where it is spent.  A nation’s wealth only accounts for 60 percent of its school system’s performance.

The top-ranking countries do not spend the most for education, but they spend a good portion on recruiting and sustaining the best teachers.  They boast of good salaries and high prestige for the teaching profession. The governments of China and Japan put substantial subsidies on teachers’ pay.

The top-ranking school systems are also serious about teacher development.

In fact, in Singapore, only the top one-third of the graduating class in college can consider taking up an education degree and only one of eight candidates is accepted in the teaching profession. Teachers there are required to train for 240 hours.

In China, teachers have to participate in 360 hours of professional development each year. Repeat—360 hours!

After recruiting smart people into the teaching profession, the top countries exert great incentives to keep them in schools by making the job not only financially attractive, but also intellectually attractive.

In Singapore, teachers learn with and from peers. It has a strong system to recruit, train, monitor and develop teachers.

Another factor in education quality is investing in resources that can make the most significant difference. In Shanghai, cooperative consortiums are set up for the best schools to help out the poorer ones, and to capitalize on the strengths of the best teachers.

Many of the top-ranking countries attract, through higher compensation, the most talented teachers to the most challenging classrooms.

“Accountability without consequences makes no sense,” Schleicher said.  So, what happens to the poor-performing teachers?

In Japan, underperforming teachers are pulled out from classrooms for a year of retraining. In China, teachersneed to go through teacher certification every five years. Those who fail in their performances have to undergo full-time training, like the pre-service teachers.

Practical advice

At first impression, Schleicher’s statistics looked mind-boggling, and the implications, overwhelming. But Schleicher, who can talk about any of the 74 countries at a click of his Powerpoint, was able to translate effectively the statistics to pragmatic suggestions for all stakeholders.  He offered the following reform trajectories:

– We need to turn the belief that “only a few can achieve excellence” to the commitment that “all studentsneed to learn at high level.”

– Instead of teaching for rote learning, schools must teach students “meta-cognition,” or learning to learn, and to understand complex ways of thinking and knowing.  We are not only teaching a subject, but teaching a student.

– Teachers must achieve a high level of professionalism as knowledge workers and researchers by “looking outward” to peers and stakeholders for continuous learning that will improve teaching.

Implications for Filipinos

As an educator, I found Schleicher’s presentation to be excellent—informative, comprehensive, stimulating and future-forward. Regretfully, it should have been presented to government and Department of Educationofficials. Incidentally, at least half of the audience was composed of educators from foreign countries.

I have observed a lack of enthusiasm among Filipino students for math and statistics. What is disturbing is that students take this inadequacy in math rather too lightly, giving the usual alibi that “math does not like me.”

Schleicher suggested this attitude might have come from the misconception that “if I’m not good on things that can be measured, then I must be good in things that cannot be measured”—which was absolutely false.

Schleicher also said the problem of Asian schools being focused on teaching for the tests was a challenge because of pressures for high-stakes examinations in East Asia. Hence, it was imperative to understand that teaching more of the same was not the answer.

The Philippines’ performance in any kind of international educational testing has not been good at all. Conveniently, this result is often blamed on poverty. But many poorer countries are looking for ways to improve. India and Pakistan, for examples, participate in PISA.

Often, there is hesitancy on the part of our education authorities to accept the results of internationally recognized tests at face value. But by playing down the importance of these assessments, we are just delaying the hard work of a systematic investigation to find out the causes and to ask for accountability.

If we are not honest in understanding our strengths and weaknesses, we are denying our next generation of students a quality education that they deserve.

As Schleicher put it, “Students who are not prepared have diminishing life’s chances. Filipinos are not competing with fellow Filipinos; they are competing with the world. Learn from other countries. Use assessment and you will find out what you want to achieve is possible.”

(Grace Shangkuan Koo, Ph.D., is associate professor of educational psychology at the University of the Philippines. E-mail her at grace@koo.org.)