PISA for Development

by Erik Solheim
Chair of the Development Assistance Committee, Development Co-operation Directorate

PISA for Development, launched on the 27th of March 2015 in Guatemala, provides an opportunity to improve the quality of education in Guatemala and throughout the developing world. Participation in PISA will help Guatemala benchmark its educational system and support the government efforts to improve education at all levels, particularly for the most disadvantaged and the large indigenous population of the country. PISA for Development also aims to identify ways to better measure, benchmark and improve education in developing countries.

Extreme poverty has been halved in two decades, and the world is now richer, better educated and more peaceful than at any other point in human history. More than 9 out of 10 children, and almost as many girls as boys, now go to school. Guatemala has made great strides in the last few years. Participation in primary education increased from 86% in 2001 to 93% in 2011, above the world average. But we must get to 10 and make sure that children learn more.

Enrolling all children, keeping them in school and providing a good education is essential for development. The education a country has today is the economy it will have tomorrow. All the great success stories in recent times have put education at the core of development. South Korea went from one of the poorest countries in the world to one of the richest, by focusing on education and industrialisation. Young Koreans are now 390 times richer than their grandparents were. Korean 15-year olds perform better in school than any other OECD nations.

Leadership is important. The founder of Singapore, Lee Kuan Yew, emphasised the importance of education during his entire life, and the results are now visible. In Guatemala, Minister of Education del Aguila has been a champion for PISA and education reform. A former teacher, Minister del Aguila combines real experience in education with political leadership.

Huge strides have been made in securing basic education for all. Nevertheless, children are not learning enough at school. Too many students still drop out before high school or university. Shortage of skilled labour is a big problem in many developing countries. The quality of education will be a central aspect of the new United Nations sustainable development goals, which will be agreed upon later this year. The OECD has launched PISA for Development as a contribution to improve the quality of education all over the globe. Since 1997, PISA has been the leading reference on the quality of education systems worldwide. The “PISA-shock”, or the understanding that you have a lot to learn from others, has inspired many countries to reconsider policies and improve their education systems. Now the time has come to take this success to a global level and work towards better quality education for all.

The OECD has partnered with Zambia, Ecuador, Paraguay, Cambodia, Senegal and Guatemala to identify how PISA can better measure the quality of education in developing countries. The PISA test instruments will need to be adjusted to account for bigger differences between the highest and lowest performing students. New methods must be tested to evaluate students that are not attending formal schooling. We hope that these efforts will enable more countries to use PISA to set national learning policies and monitor progress.

Monitoring and evaluation is crucial for improving education. PISA is a powerful tool because it gives countries an honest assessment of whether their students are on the right track. Brazil has done more than any other PISA participant to improve its education system. Brazil was on the bottom of the ranking when it first participated in 2000. Then Brazil used PISA to prioritise policies and do focus more on what policies work better. Brazil improved the quality of its education system faster than any other nation over the next 10 years. The Bolsa Familia program, which provided cash remuneration to low-income families in exchange for enrolling children in school also contributed to improved education for the poorest children most likely to be out of school.

PISA opens up opportunities to allow us to learn from the best. Chinese and other East Asian students are the best performers in mathematics and science, and reading. The seven top spots on the PISA ranking are occupied by Asian countries and cities. There must be many things we can learn from Asian success stories, whether the secret is aspiring students, good teachers or better public policies. Guatemala and Peru is paired as part of a mentoring mechanism to provide peer-to-peer technical advice and learn from each other. Identifying policies that work and implementing these on a global scale is key for improving the quality of education.

Good policies are much more important than money.  Fifteen year old students in poor Vietnam are doing better than the average student in much richer countries. Only about 6% of performance differences in PISA test are explained by national income. Money is best used to underpin good policies.

Quality education is the way to development and poverty reduction. Monitoring progress, learning from success stories and implementing the best policies is the way to improve quality of education. Let the children learn!

Links:
PISA in Development
PISA 2012 Key Findings
Photo credit: © Roberto Franco Arias/Pedro Molina

A chance to design the way forward for education

by Michael Ward
Senior Policy Analyst, the Development Co-operation Directorate

Want to get involved in shaping the future of education? As the United Nations Millennium Development Goals (MDG) reach their 2015 deadline, several international groups, including the OECD, are formulating a new set of goals and targets for sustainable development… and we’d like to know what you think.

The Open Working Group on Sustainable Development Goals (OWG), a UN-appointed task force, has proposed an agenda for development that includes goals for education, and educators from around the world have developed a set of specific education and learning targets that are closely aligned with that agenda.

The task now is to develop indicators so that progress towards achieving these new goals can be monitored.

To that end, a Technical Advisory Group, co-ordinated by the UNESCO Institute for Statistics and including members from the Education for All Global Monitoring Report, the OECD, UNESCO, UNICEF and the World Bank, has proposed a set of indicators, which you can find here.

We’d like to hear from you

What do you think about the indicators?

Between today and 30 January 2015 you are invited to comment on each indicator or to respond to these questions:

1. For each target, does the report identify the best indicators that are most aligned with the concept and are already being tracked in a large number of countries?

2. What new indicators could be developed to be more closely aligned with the proposed targets and have the potential to be globally comparable?

3. For each target, please identify or propose the two most important indicators.

4. Are there key issues that the document has not addressed satisfactorily or other issues that also need to be taken into consideration?

Please visit the UIS website for details on how to submit your comments.

To ensure that the consultation is open to as many people as possible, we invite you to spread the word among your networks and social media, referencing #Education2015.

After 30 January 2015, the Technical Advisory Group will review the list of indicators based on your feedback. The final proposal will be submitted for endorsement at the World Education Forum in Incheon (Korea) in May 2015. The final documents will then form the basis of the discussions at the UN General Assembly in September 2015 on the new UN goals for education.

Join the discussion on twitter via @EFAReport @UNESCO @WBEducation @OECD_Edu @UNICEFEducation and #Education2015.

Links:
OECD PISA for Development
Photo credit: ©UNESCO Institute for Statistics (UIS)

Expanding PISA’s circle of influence (part two)

by Barbara Ischinger, Director, OECD Directorate for Education and Skills
Michael Ward, Senior Policy Analyst, PISA for Development
and Alejandro Gomez Palma, Policy Analyst, PISA for Development

In our previous blog about PISA for Development, we were pleased to announce Ecuador’s agreement to participate in this new pilot project. We’ve just returned from the Zambian capital of Lusaka and are delighted to report that Zambia has also agreed to participate – the first sub-Saharan African nation ever to take part in any PISA survey.

You might well ask: how can we compare the performance of students in highly developed countries – such as Japan and Germany – with that of students in low- and middle-income countries in Africa? And how do we assess the competencies of the tens of millions of 15-year-olds in developing countries who aren’t enrolled in school? These were precisely some of the challenges we put to ourselves when the idea of PISA for Development emerged from discussions with countries and OECD partners.

To make the assessments of 15-year-olds more relevant to a wider range of countries, including Zambia, we knew that we would have to adapt both the tests and the contextual information we collect through PISA’s student, school and parent questionnaires. For the assessment of reading, mathematics and science, we will tap into our large stock of questions that were already successfully used in previous PISA cycles and, in collaboration with our partners, select, review and adapt and then field trial those that we find are best suited to assess the level of skills we identify in the participating countries. The PISA for Development assessments will be better targeted to describe the performance at the middle and low end of the proficiency spectrum, while also measuring the higher levels and maintaining comparability with the international PISA scales. The range of questions included in the assessment will give us a fine-grained picture of performance at these lower levels that will, in turn, provide more valuable diagnostic information for countries. 

The questions we select will be field tested in Zambia and the other participating countries to see how well students respond to them, then reviewed and adapted, as necessary, with the assistance of local and regional experts, to ensure they are relevant across countries and cultures. This adaptation may involve making the questions more familiar to students by changing choices of words and references, for example, and, where necessary, translating the questions into the languages of the participating countries. The translation and verification process is well established in the main PISA assessment and it also allows for some adaptations to accommodate local contexts. There are, for example, different ways of asking the same question depending on whether the student taking the test lives in Australia or in the United Kingdom; and Scotland also adapts some of the terminology of some of the PISA test questions. But regardless of these contextual changes, the nature of the questions, and the knowledge and skills assessed, must remain true to the original intent of the question, so that the assessments still measure what they were originally designed to measure and results are comparable with those of other countries.

The context questionnaires, distributed to students, school directors and parents, will be similar to those we already use in the main PISA assessment, but will be developed to include questions that may be more relevant to countries like Zambia. So, for example, where the current PISA questionnaire to school directors asks about school facilities and resources, this would need to be adapted to include such questions as: Does the school have separate sanitation facilities for girls and boys? Does the student have electricity at home? Running water? A separate kitchen? And we will have to be prepared for very different answers to some basic questions – like: who is the head of your household? – as households in developing countries are sometimes headed by children or grandparents. An important point to remember is that we have to keep some questions common to all so that we can assess all countries on the same scale.

Field trials of the PISA for Development assessment will begin in the second half of 2015; we’re expecting that more than 25,000 students in 4 or 5 countries – including Ecuador and Zambia – will participate. The main assessment will be conducted during 2016 and early 2017. During this time, we’ll also continue working with our partners, including the World Bank, UNICEF and UNESCO, to try to identify 15-year-olds who are not enrolled in school and develop ways of reaching them, too. It’s an ambitious agenda, certainly. But we’re convinced that, as the world moves beyond the Millennium Development Goals towards a new post-2015 education agenda that is focused on improving learning outcomes worldwide, we must be sure that children in all countries not only have access to school, but are acquiring both fundamental and 21st century skills when they get there.

Links:
Expanding PISA’s circle of influence
PISA for Development
Millennium Development Goals
Photo Credit: Drawing of Zambia on Blackboard, drawn in chalk / @Shutterstock

Expanding PISA’s circle of influence

by Barbara Ischinger, Director, OECD Directorate for Education and Skills
and Alejandro Gomez Palma, Analyst, PISA for Development

The enormous worldwide interest in the PISA 2012 results, which were released last December, showed that PISA is now widely accepted as the best measure of student performance we have – and one of the best sources of data that can be used to inform policy decisions about how to improve education systems. Sixty-five countries and economies participated in PISA 2012, but that leaves well over 100 others that either chose not to or believe that participation is out of their reach. We hope that that’s going to change soon.

We’ve just returned from Ecuador where the government agreed – with a sense of pride that was palpable – to participate in our pilot PISA for Development initiative. Eight Latin American countries participated in the latest round of PISA 2012 and we’re keen to add to that number. We’ve briefly mentioned the PISA for Development project in earlier blogs, but now that countries are signing on, we want to describe in more detail what it is and what it means, both for the OECD and for the countries involved.

The OECD has already done a lot of work on development issues (among other things, we helped to define the United Nations Millennium Development Goals, for example). We have often been asked to conduct reviews of education systems in developing countries and emerging economies; but because we haven’t always had a good evidence base for these countries, we didn’t feel we could provide the kind of analysis they were looking for.

In many of these countries, the main obstacle to conducting such reviews is the absence of good data. There is hardly any information at all on education outcomes, including results of student assessments or the level of quality and equity of the education provided. While some assessments and evaluations of education are conducted at the regional level in many areas, including Latin America and sub-Saharan Africa, and we have been working with – and learning from – the organisers of these assessments as we shape PISA for Development, they cannot offer the kind of global benchmarking that PISA can.

The data PISA elicits is not just used to develop “league tables” comparing countries’ performance; it can – and should – be used as a public policy tool: governments can release the information to inform a public debate on how to improve the quality of their education systems.

A lot of what we will be doing in PISA for Development is building capacity in participating countries for managing large-scale student assessments. For the countries participating, our aim is to bring the level of capacity up to the minimum level needed to implement PISA.  In fact, many developing countries that have already participated in PISA have said that one of the biggest benefits has been the attendant capacity-building that is part of PISA.

The PISA for Development project fits in nicely with work on the post-2015 agenda, the successor framework to the Millennium Development Goals, which are likely to focus on ensuring access to education plus improving the quality of education, particularly the quality and equity of learning outcomes. How is the developing world going to measure quality in education? Well, the OECD and the over 70 countries that have participated in PISA since the first round in 2000 would respond that we already have a proven instrument: PISA. What we need to do now is to modify the assessment so that it can be used by a far larger set of countries that have a wider spectrum of student performance.

We’ll tell you how we’re adapting the PISA instruments, methods and analyses for PISA for Development – and targeting countries in sub-Saharan Africa, too, none of which has ever participated in PISA – in a subsequent blog.

Links:
PISA 2012 results
PISA for Development
United Nations Millennium Development Goals
Latin American Laboratory for Assessment of the Quality of Education (LLECE)
CONFEMEN Programme for the Analysis of Education Systems (PASEC)
The Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ)
Photo Credit: INEVAL/Ecuador