Students at the centre: promoting effective evaluation and assessment in Northern Ireland

by Claire Shewbridge
Analyst, Early Childhood and Schools Division, Directorate for Education and Skills

“Students do not want to be ‘passive objects’ of evaluation and assessment, they want to be actively involved”. When we heard the representative of the European School Student Unions say this at a conference in April, we smiled. We had just returned from eight days in Northern Ireland where we’d been really impressed with how much students knew about their assessment. Of course, we only got the chance to visit a few schools, but although each school had a distinct approach to assessment, they all shared a commitment to getting the students involved.

As explored in a new OECD report on evaluation and assessment policies in Northern Ireland, current policy, together with the curriculum, promotes the engagement of students in their own evaluation by encouraging them to talk about, review and make improvements to their work, as well as to ask questions and to respond to others’ points of view. This allows students to develop ‘higher order skills’ such as meta-cognitive awareness: the ability to reflect and analyse the learning experience itself. Engagement is also important for school inspectors, who check to what extent schools are using a broad range of assessment policies and engaging students in self- and peer-assessment.

But in addition to evaluating themselves, students want to be involved in the evaluation of their teachers and schools. In Northern Ireland, official policy underlines the importance of school self-evaluation for improving the learning experience for students, and evidence from school inspections indicates that self-evaluation activities are highly developed in many schools. This is backed up by new PISA 2012 results, which indicate that 65% of 15-year-olds in Northern Ireland are in schools that give students an opportunity to provide written feedback on teachers, lessons and resources, compared with an OECD average of 61%. 

The evaluation of schools in Northern Ireland can play a crucial role in addressing the existing equity challenges by monitoring whether effective equitable and quality education is being provided. This is a key factor in increasing the social mobility of children and young people from disadvantaged backgrounds, and addressing economic and social challenges. In this way, evaluation promotes improvement for ALL students.

Going forward, Northern Ireland will need to build public awareness and further develop professionalism in schools, ensuring that policies are reviewed and refined where necessary. But throughout all of this, the focus of evaluation should always be kept on the improvement of student learning outcomes, and on involving students. With this, evaluation and assessment become more effective for schools and governments, and more relevant for students.

Access OECD Reviews of Evaluation and Assessment in Education for Northern Ireland, United Kingdom and the other participating countries, along with the final project report, ‘Synergies for Better Learning: an International Perspective on Evaluation and Assessment’ on the OECD website Reviews on Evaluation and Assessment Frameworks for Improving School Outcomes. 

Links:
Synergies for better learning 
PISA 2012 Results: What Makes Schools Successful? Resources, Policies and Practices, Volume IV
Related blog post Evaluation and assessment is for everyone


Photo credit: student assessment @ Shutterstock

Are the Chinese cheating in PISA or are we cheating ourselves?

by Andreas Schleicher
Deputy Director for Education and Skills and Special Advisor on Education Policy to the OECD's Secretary General

Whenever an American or European wins an Olympic gold medal, we cheer them as heroes. When a Chinese does, the first reflex seems to be that they must have been doping; or if that’s taking it too far, that it must have been the result of inhumane training.

There seem to be parallels to this in education. Only hours after results from the latest PISA assessment showed Shanghai’s school system leading the field, Time magazine concluded the Chinese must have been cheating. They didn't bother to read the PISA 2012 Technical Background Annex, which shows there was no cheating, whatsoever, involved. Nor did they speak with the experts who had drawn the samples or with the international auditors who had carefully reviewed and validated the sample for Shanghai and those of other countries.

Others were quick to suggest that resident internal migrants might not be covered by Shanghai’s PISA sample, because years ago those migrants wouldn't have had access to Shanghai’s schools. But, like many things in China, that has long changed and, as described by PISA, resident migrants were covered by the PISA samples in exactly the way they are covered in other countries and education systems. Still, it seems to be easier to cling to old stereotypes than keep up with changes on the ground (or to read the PISA report).

True, like other emerging economies, Shanghai is still building its education system and not every 15-year-old makes it yet to high school. As a result of this and other factors, the PISA 2012 sample covers only 79% of the 15-year-olds in Shanghai. But that is far from unique. Even the United States, the country with the longest track record of universal high-school education, covered less than 90% of its 15-year-olds in PISA - and it didn't include Puerto Rico in its PISA sample, a territory that is unlikely to have pulled up U.S. average performance.

International comparisons are never easy and they are never perfect. But anyone who takes a serious look at the facts and figures will concede that the samples used for PISA result in robust and internationally comparable data. They have been carefully designed and validated to be fit for purpose in collaboration with the world’s leading experts, and the tests are administered under strict and internationally comparable conditions. Anyone who really wants to find out can review the underlying data.

Short of arguments about methodology, some people turn to dismissing Shanghai’s strong performance by saying that Shanghai’s students are only good on the kind of tasks that are easy to teach and easy to test, and that those things are losing in relevance because they are also the kind of things that are easy to digitise, automate and outsource. But while the latter is true, the former is not. Consider this: Only 2% of American 15-year-olds and 3% of European ones reach the highest level of math performance in PISA, demonstrating that they can conceptualise, generalise and use math based on their investigations and apply their knowledge in novel contexts. In Shanghai it is over 30%. Educators in Shanghai have simply understood that the world economy will pay an ever-rising premium on excellence and no longer value people for what they know, but for what they can do with what they know.

PISA didn't just test what 15-year-olds know in mathematics, it also asked them what they believe makes them succeed. In many countries, students were quick to blame everyone but themselves: More than three-quarters of the students in France, an average performer on the PISA test, said the course material was simply too hard, two-thirds said the teacher did not get students interested in the material, and half said their teacher did not explain the concepts well or they were just unlucky. The results are very different for Shanghai. Students there believe they will succeed if they try hard and they trust their teachers to help them succeed. That tells us a lot about school education. And guess which of these two countries keeps improving and which is not? The fact that students in some countries consistently believe that achievement is mainly a product of hard work, rather than inherited intelligence, suggests that education and its social context can make a difference in instilling the values that foster success in education.

And even those who claim that the relative standing of countries in PISA mainly reflects social and cultural factors must concede that educational improvement is possible: In mathematics, countries like Brazil, Turkey, Mexico or Tunisia rose from the bottom; Italy, Portugal and the Russian Federation have advanced to the average of the industrialised world or close to it; Germany and Poland rose from average to good, and Shanghai and Singapore have moved from good to great. Indeed, of the 65 participating countries, 45 saw improvement in at least one subject area. These countries didn't change their culture, or the composition of their population, nor did they fire their teachers. They changed their education policies and practices. Learning from these countries should be our focus. We will be cheating ourselves and the children in our schools if we miss that chance.

International comparisons are never easy and they aren’t perfect. But PISA shows what is possible in education, it takes away excuses from those who are complacent, and it helps countries see themselves in the mirror of the educational results and educational opportunities delivered by the world’s leaders in education. The world has become indifferent to tradition and past reputations, unforgiving of frailty and ignorant of custom or practice. Success will go to those individuals, institutions and countries which are swift to adapt, slow to complain and open to change. And the task for governments is to help citizens rise to this challenge. PISA can help to make that happen.

Links:
PISA 2012 Results
OECD Programme for International Student Assessment
Related blog post by Andreas Schleicher: What we learn from the PISA 2012 results
Slideshare PowerPoint presentation: PISA 2012 Evaluating school systems to improve education
Press release: Asian countries top OECD's latest PISA survey on state of global education

Photo credit: Chinese olympic athlete @ Shutterstock

What we learn from the PISA 2012 results

by Andreas Schleicher
Deputy Director for Education and Skills and Special Advisor on Education Policy to the OECD's Secretary General

 

International comparisons are never easy and they aren’t perfect. But PISA shows what is possible in education, and it helps countries see themselves in the mirror of the education opportunities and results delivered by the world’s leaders in education. Even those who claim that the relative standing of countries in PISA mainly reflects social and cultural factors must now concede that improvement in education is possible. In mathematics, countries like Brazil, Mexico, Tunisia and Turkey rose from the bottom; Italy, Portugal and the Russian Federation advanced to the OECD average or close to it; Germany and Poland rose from average to good; and Shanghai-China and Singapore have moved from good to great. Indeed, of the 65 participating countries, 40 saw improvement in at least one of PISA’s three subject areas. These countries did not change their culture, or the composition of their populations, nor did they fire their teachers; they changed their education policies and practices.

We focused this year’s PISA assessment on mathematics. Each year, OECD countries invest over 200 billion euro in math education in schools; but poor math skills still severely limit people’s access to better-paying and more-rewarding jobs and, at the aggregate level, inequality in the distribution of math skills closely relates to how wealth is shared within nations.

This PISA 2012 assessment came at a time when countries were still grappling with the aftermath of the economic crisis - a period that has brought home the urgency of equipping more people with better skills to collaborate, compete and connect in ways that drive our economies forward, foster employment and reduce social inequality.

A large part of the challenge in education lies in addressing underperformance. Across countries, almost one in four 15-year-olds did not even reach Level 2, the PISA baseline level of proficiency in mathematics, where students have to do little more than employ basic algorithms or procedures involving whole numbers. But in Canada, Korea, Shanghai-China and Singapore, it is one in ten or fewer. According to one estimate, if all 15-year-olds in the OECD area attained at least PISA Level 2 in math, they would contribute USD 200 trillion in additional economic output over their working lives. While such estimates are never wholly certain, they do suggest that the benefits of improvement dwarf any conceivable cost. Part of the issue lies with students living in social disadvantage, and many school systems amplify that disadvantage. According to PISA, advantaged and disadvantaged schools show particularly wide differences in levels of teacher shortages. Attracting the most talented teachers and school leaders to the most challenging classrooms will therefore be key to making headway. Indeed, PISA finds that higher-performing countries allocate educational resources more equitably among advantaged and disadvantaged schools.

A belief that all students can achieve at a high level and a willingness to engage all stakeholders in education – including students, through such channels as seeking student feedback on teaching practices – are other hallmarks of successful school systems. New results from PISA also show that students whose parents have high expectations for them tend to have more perseverance, greater intrinsic motivation to learn math, and more confidence in their own ability to solve math problems.

But the challenges of school systems are not just about poor kids in poor neighbourhoods, but about many kids in many neighbourhoods. Only 2% of American students reach the highest level of math performance, demonstrating that they can conceptualise, generalise and use math based on their investigations and apply their knowledge in novel contexts. That compares with an OECD average of 3%, and proportions of up to 31% in Shanghai-China. The world economy will pay an ever-rising premium on excellence, and a number of countries have shown how the share of top performers in school can be raised significantly, including in high performers, such as Hong Kong-China and Korea, and low performers, such as Italy, Portugal and the Russian Federation. It is important that raising excellence and improving equity are not seen as conflicting policy objectives. Indeed, of the 13 countries that significantly improved their math performance since 2003, three also show improvements in equity in education, and another nine improved their performance while maintaining an already high level of equity.

Of course, raising outcomes is easier said than done. The status quo has many protectors, and countries need to be bold in thinking and in execution to effect real changes. Obviously, we can’t copy and paste school systems wholesale. But PISA has revealed an encouraging number of features shared by the world’s most successful school systems.

Everybody agrees education is important. But the test comes when education is weighed against other priorities. How do countries pay their teachers, compared to other highly skilled workers? Would you want your child to be a teacher rather than a lawyer? How do the media talk about teachers? What we’ve learned from PISA is that the leaders in high performing systems have convinced their citizens to make choices that value education, their future, more than consumption today.

But placing a high value on education is just part of the equation. Another part is the belief that all children can achieve. The fact that students in some countries consistently believe that achievement is mainly a product of hard work, rather than inherited intelligence, suggests that education and its social context can make a difference in instilling the values that foster success in education.
In the past, different students were taught in similar ways. Today’s top school systems embrace diversity with differentiated instructional practices; they realise that ordinary students have extraordinary talents and they personalise educational experiences. High-performing school systems also share clear and ambitious standards across the board. Everyone knows what is required to get a given qualification. This remains one of the most powerful system-level predictors in PISA.

And nowhere does the quality of a school system exceed the quality of its teachers. Top school systems pay attention to how they select and train their staff. They watch how they improve the performance of teachers who are struggling and how to structure teachers’ pay. They provide an environment in which teachers work together to frame good practice. And when deciding where to invest, they prioritise the quality of teachers over the size of classes. Not least, they provide intelligent pathways for teachers to grow in their careers.

High performers have also moved on from administrative control and accountability to professional forms of accountability and work organisation. They support their teachers in developing innovations in pedagogy, in improving their own performance and that of their colleagues, and in pursuing professional development that leads to stronger education practice. The goal of the past was standardisation and compliance; now, top performers enable teachers to be inventive. In the past, the policy focus was on providing education; in today’s top school systems, it’s on outcomes, shifting from looking upwards in the bureaucracy towards looking outwards to the next teacher, the next school, about creating networks of innovation.

Perhaps the most important outcome of world-class school systems is that they deliver high-quality education across the entire school system so that every student benefits. Overall, Finland did not come out quite as impressively as in previous assessments; but what makes Finland still special is that only 6% of the performance variation among students lies between schools. In other words: every school succeeds.

Last but not least, high-performing systems tend to align policies and practices across all aspects of the system, they make them coherent over sustained periods of time, and they see that they are consistently implemented.

Of course, there is no single combination of policies and practices that will work for everyone, everywhere. Every country has room for improvement, even the top performers. That’s why the OECD produces this triennial report on the state of education across the globe: to share evidence of the best policies and practices and to offer our timely and targeted support to help countries provide the best education possible for all of their students. With high levels of youth unemployment, rising inequality, a significant gender gap, and an urgent need to boost growth in many countries, we have no time to lose.

Links:
PISA 2012 Results
OECD Programme for International Student Assessment
Press release: Asian countries top OECD's latest PISA survey on state of global education
Follow:
PISA on twitter: @OECD_Edu @SchleicherEdu @OECDLive #OECDPISA
Facebook PISA  for Parents