More data for better policies

by Dirk van Damme
Head of the Innovation and Measuring Progress division, Directorate for Education and Skills

As recently as 30 years ago, politicians, leaders and practitioners believed that all economic and social systems could and should be measured, and that managing these systems better would require more data. Except for education systems. Education systems were considered to be so different across countries that international data would never do justice to each system’s specificity. And what happened in the classroom was something believed to be unmeasurable. Yes, maybe you could count the number of students or calculate the years people spend in school; but that was basically it.

Some pioneers had the courage to think differently. Against the tide, and confronting a lot of resistance, they organised international network meetings to discuss the essence of what was happening in education, agree on definitions, develop measurement tools, and exchange and compare data. After all, how could it be that what seemed to be so evident in many other complex systems was impossible in education?

Back to the present. The pioneers have gone; highly professional teams are now bringing their ideas into fruition. We have come to understand that, just as in all other spheres of life, many dimensions of what happens in education can be measured and assessed in an internationally comparative way, without doing injustice to the complexity and sensitivity of education. We now know not only how to count students and the amount of money invested in education, we also know how to translate complex realities into accessible language, thanks to tools such as ISCED. The Programme for International Student Assessment (PISA) has taught us how to master the challenge of assessing the knowledge and skills that students acquire in schools. We have been able to relate education to external data on employment, earnings, health outcomes and more intangible outcomes, such as interpersonal trust, to better understand the many social purposes that education serves.

On 9 September the OECD will publish its 2014 edition of Education at a Glance, the world’s most extensive and authoritative data source of international educational statistics and indicators. This year’s edition includes no less than 30 different indicators and over 100 000 pieces of data. Policy makers, education leaders, practitioners and a wide variety of stakeholders will try to find out how their system is doing compared to that of other countries. How educated have our societies become? Have the investments in our schools decreased as a consequence of the economic crisis? How large is the earnings premium from which tertiary graduates benefit over their lifetime? Do pupils in private schools perform better than in public schools? Is social mobility a reality or just an aspiration? How well do we pay our teachers? How many students now travel over the globe to study elsewhere? These questions emerge not just out of curiosity. The data and the evidence increasingly serve to underpin and improve policies. A system as complex as education cannot be managed and steered without reliable and comparable data. Better and more data result in better policies.

This year’s edition is particularly rich because the OECD has been able to benefit from three main sources of survey data: the PISA 2012 database on learning outcomes of 15-year-olds, the data from the Survey of Adult Skills (a product of the OECD Programme for the International Assessment of Adult Competencies, or PIAAC), and data on teachers from the 2013 OECD Teaching and Learning International Survey (TALIS). Combined with the data collected from administrative sources, we now have the largest database in education ever brought together in human history.

Does this mean that the mystery of the educational encounter between teacher and learner has been sacrificed on the altar of numbers? No, just as the sophisticated data-monitoring systems in the health sector have not destroyed the genius of the medical act. In both cases the data have helped to create better conditions for the magic to happen.

Launch events:
Live streaming of launch event in Brussels with Andreas Schleicher, OECD Director of Education and Skills, and Xavier Prats Monné, European Commission Deputy Director-General for Education and Culture
OECD Education and Skills webinar presenting Education at a Glance: 2014 OECD Indicators (registration required. Password: OECDEDU)
Follow #OECDEAG on Twitter:  @OECD_Edu @OECDLive @EAG_Indicators
See full listing of media events
Links:
OECD Education at a Glance
Education GPS - the OECD source for internationally comparable data on education policies and practices
Photo credit: © OECD

What can we learn from our teachers?

by Kristen Weatherby
Senior Analyst, Directorate for Education and Skills

 

The latest results from the OECD Teaching and Learning International Survey (TALIS) are made public today at various events in countries around the world. TALIS 2013 surveyed 107,000 lower secondary school teachers in 34 countries.  Everyone from education ministers – who are gathered at an event in Tokyo – to teachers – like those at a TALIS conference in Madrid – want to learn from the data collected in the survey in order to improve the teaching and learning in their schools.

So what are teachers telling us? First of all, teachers love being teachers. On average across TALIS-participating countries, 9 in 10 teachers report being satisfied with their jobs, and nearly 8 in 10 (78%) report that they would still choose to become a teacher if they had to make the choice again.

Given this finding, it is perhaps surprising that, on average, more than two out of three teachers across TALIS countries do not feel that their profession is valued by society. This percentage varies by country: in some countries, particularly those with high-performing education systems (Finland, Korea, Singapore), notably larger proportions of teachers report feeling that their profession is indeed valued by society.

Why do most teachers feel that teaching isn’t valued? And why does it matter? In some countries, it could be that the teachers’ perceptions are correct, and that societies may not value teaching as much as other professions. But it could also have something to do with how teaching has evolved – or not – as a profession. If you take a look at TALIS 2013 Results: An International Perspective on Teaching and Learning, you will learn a great deal about what teachers say about their background, education, support, development and teaching practices. Together with data from school principals, these data paint a picture of the teaching profession around the world today.

When you look at the statistics on teacher appraisal and feedback, for example, it’s not difficult to see why some teachers may not feel valued. The teachers surveyed agree that appraisals are helpful, with more than 6 in 10 teachers reporting that appraisal leads to positive changes in their teaching practices. Yet nearly half of teachers feel that the appraisals in their school are performed simply to fulfil administrative requirements. Only about one in three teachers feels that the feedback received will lead to any kind of career advancement, which might include higher pay or additional responsibilities. Indeed, nearly 80% of teachers report that annual increments in their salaries are awarded regardless of the outcome of formal teacher appraisals.

If we want teaching to improve so that school systems can produce the skilled citizens that our societies need, then we not only need to change the practices of existing teachers, we also need to ensure that teaching attracts high-quality candidates. Providing teachers with a career path that includes recognition for good performance and support to improve is certainly one way to start. TALIS data also indicate that teachers who are given the opportunity to participate in decision making at school not only are more likely to report that teaching is valued as a profession, they also report higher job satisfaction and more confidence in their own abilities as teachers. Thus it seems about time to treat teachers as the professionals they are.

Links:
2013 TALIS Results
Free Teachers’ Guide to TALIS
Join:
Education Fast Forward (EFF10) Global live debate - 25 June 2014
Alliance for Excellent Education webinar - 27 June 2014
Photo credit: © Fotolia

Who are our teachers?

by Kristen Weatherby

Senior Analyst, Teaching and Learning International Survey (TALIS)

As we celebrate this year’s UNESCO World Teachers Day, many of us think back to our favourite teacher. Mine was Mr. Monroe, the high school English teacher who instilled in me a love for writing that still exists today.  We all have favourite teachers, those inspirational leaders whom we hope our children or loved ones will encounter at some point during their schooling. But what makes a good teacher? And what do we know about the teachers in our schools today?

When the results of the OECD’s Teaching and Learning International Survey were first released in 2009, we were able to start answering some of these questions. TALIS 2008 was the first and largest international survey of teachers, giving teachers and school leaders in 24 countries a voice to speak about their experiences. Teachers told us about their initial teacher training and the professional development they receive; the feedback they get on their teaching; the climate in their classrooms and schools; their own satisfaction with their jobs; and their feelings about their abilities as teachers.

In the past four years, TALIS findings and cross-country comparisons have informed policy recommendations, been at the heart of ministerial-level discussions at the International Summits on the Teaching Profession, and driven discussions about the future of teaching as a profession. As more countries are now talking about improving teacher quality, TALIS data is more relevant than ever.

The OECD just finished data collection for the current cycle of TALIS, which has expanded to include 33 countries. As with TALIS 2008, all countries surveyed lower-secondary teachers, but several countries also chose to study their primary and upper-secondary school teachers as well. We have a team of analysts and experts looking at the data now, and will reveal the results of TALIS 2013 in June of 2014.

We’re looking forward to getting an updated picture on the state of teaching in our schools, as well as some new information. Watch this space over the coming months to learn more about how you can access the new data.

Until then, take a look at our newly-updated TALIS website, where you can easily find all of our publications, policy briefs, TALIS 2008 data, and the questionnaires we use in the survey.  And take some time to say thank you to your favourite teacher today. Thank you, Mr. Monroe!

Links
OECD Teaching and Learning International Survey 
World Teachers' Day 2013 celebrations at UNESCO 
Teachers for the 21st Century: Using Evaluation to Improve Teaching
Synergies for Better Learning: An International Perspective on Evaluation and Assessment
Follow TALIS and Kristen Weatherby @Kristen_Talis
Photo credit: Question mark symbol in chalk over complex calculation /@ Shutterstock

Education GPS: A world of education at your fingertips

by Jean Yip,
Analyst, Directorate for Education and Skills

Today the OECD launched the Education GPS, the source for internationally comparable data on education policies and practices, opportunities and outcomes. Accessible any time, in real time, this platform gives you the latest data and analysis of countries’ performance in providing high-quality education for all.

Analyse by country

The Education GPS enables you to find, use and understand data and analysis on individual countries. Choose from a variety of themes, explore our country profiles, and create your own customised country reports. 

Explore data

The Education GPS gives you easy access to the OECD’s data on education. Choose a theme, find the related data, and compare countries.

Review education policies (coming soon...)

The Education GPS provides access to OECD’s extensive research and analysis of education policy around the world. See how education policies complement or compete with each other, and which policies the OECD recommends. 

Try it for yourself and explore the wealth of OECD’s data and analysis of education policies and practices!

Links:
Education GPS 

What a tangled web we weave: strategies for school improvement

by Harald Wilkoszewski
Analyst, Innovation and Measuring Progress Division, Directorate for Education and Skills

The best education possible for our children – this is what all education systems strive for. But what can policy makers do when schools become weak, sometimes too weak, to fulfil their purpose? Especially in highly decentralised systems, where schools enjoy high levels of autonomy, policy interventions to help improve the learning situations for students can be particularly difficult to implement.

A new case study on The Netherlands for the OECD project “Governing Complex Education Systems (GCES)” shows that a timely, risk-based assessment of schools can help to significantly lower the number of weak schools. The case study also illustrates that governing is in the details, as the reform had some undesired effects. But let’s look at the good news first.

In 2009, 120 Dutch schools were underperforming. In 2011 this number decreased to less than 100, a significant drop of over 16%. The Dutch school inspectorate, traditionally responsible for school quality, achieved this by flagging schools that showed low indicators on a number of outputs as “weak” or “very weak”. These labels mean that unless these schools change radically, they run the risk of failing – being closed down.

Schools are not, however, left alone with this message: The inspectorate puts in place a range of supporting measures such as tailor-made recommendations or an intensive exchange with schools boards.

While the numbers speak for the success of this reform, the new GCES case study “Coping with very weak primary schools” also shows that schools react quite differently to being labelled “weak” or “very weak”. Most school communities take it as a wake-up call and are encouraged to improve, but some become discouraged and do not manage to turn the wheel around.

Whereas schools that react positively then end up in a virtuous cycle of improvement, schools that react negatively are stuck in a vicious circle: alerted parents take their children out of the labelled school, other parents follow the example, and the number of students decreases dramatically. This forces the school to merge several school grades into one classroom, putting even more stress on the teachers. In this difficult situation, performance improvements are hard to achieve and school closure becomes a likely scenario.

In order to avoid such undesired effects, the case study recommends that decision makers adapt their mindsets, from a linear perspective of cause and consequence, to a more dynamic view. This will help them take a second look at their existing repertoire and analyse how their current set of policy interventions works in a complex system.

Ultimately, this adaptive view may prove beneficial in other education policy contexts as well because it can:

• Lead to new insights into how and why some interventions seem to work surprisingly well;
• Show why some methods work less well and identify previously unnoticed, positive effects;
• Show that some interventions do not work at all and that the capital that goes into them may be better invested in other areas of the system;
• Put some interventions on the radar that come from other areas, other policy domains that may be beneficial to very weak schools;
• Be an important element in developing smarter interventions in multi-level systems.

Links:
OECD'S Centre for Educational Research and Innovation (CERI)
Governing Complex Education Systems (GCES)
Photo credit: Young girl excited that she has reached the top of the giant/ Shutterstock

Big data and PISA

by Andreas Schleicher
Deputy Director and Special Advisor on Education Policy to the OECD's Secretary-General

Big data is the foundation on which education can reinvent its business model and build the coalition of governments, businesses, and social entrepreneurs that can bring together the evidence, innovation and resources to make lifelong learning a reality for all. So the next educational superpower might be the one that can combine the hierarchy of institutions with the power of collaborative information flows and social networks. More than anything else, this will hinge on getting people to generate innovative applications on top of big data. It’s about the co-creation of governance, about delivering more progressive and better policies than the industrial work organisation and the bureaucratic and litigation-oriented tools and strategies that we are used to in education.

This isn’t just about improved transparency and public accountability in education. Throwing education data into the public space does not change the ways in which students learn, teachers teach and schools operate. It does not lead to people doing anything with that data and transforming education in ways that will actually change education practice. On the contrary, it often results simply in adversarial relationships between civil society and government over the control and ownership of information.

The prerequisite for using big data as a catalyst to change education practice is to get out of the “read-only” mode of our societies. It’s about combining transparency with collaboration. The way in which educational institutions often work is that you have a single expert sitting somewhere in a corner who determines the application of rules and regulations affecting hundreds of thousands of students and teachers – and nobody can figure out how those decisions were made. Big data can lead to big trust if we make that data available, train civic innovators, experiment, create a maker culture. It is no surprise that OECD’s new Survey of Adult Skills shows that the more proficient people are in literacy, the more they trust others.

Collaborative consumption provides a great example of this. These days, people share their cars and even their apartments with strangers. Collaborative consumption has made people micro-entrepreneurs – and its driving engine is building trust between strangers. Think about it: in the business world, we have evolved from trusting people to provide information, to willingly handing over credit card data, to connecting trustworthy strangers in all sorts of marketplaces. We are light-years away from that when it comes to data about education.

But here’s how we can get a little closer. Some years ago we created PISA, a global survey that examines the skills of 15-year-olds in ways that are comparable across countries. PISA has created huge amounts of big data about the quality of schooling outcomes. PISA has also helped to change the balance of power in education by making public policy in the field of education more transparent and more efficient. At the micro-level, there were still a lot of sceptics: teachers thought this was just another accountability tool through which governments wanted to control them. So what did we do? This year we put in place a kind of “MyPISA” – PISA-type instruments that we circulated out into the field. The PISA based test for schools provides comparisons with other schools anywhere else in the world, schools that are similar to them or schools that are very different.

Suddenly, the dynamic has changed; schools are beginning to use that data. Ten schools in Fairfax county in Virginia, for example, have started a year-long discussion among principals and teachers based on the results of the first reports. With the help of district offices (and the OECD), they will be conducting secondary analyses to dig deeper into their data and understand how their schools compare with each other and with other schools around the world. Those principals and teachers are beginning to see themselves as teammates – not just spectators – on a global playing field. In other words, in Fairfax county, big data is building big trust.

Links:   
For more information on PISA:www.oecd.org/pisa/
OECD Survey of Adult Skills (PIAAC)
The Role of Data in Promoting Growth and Well-Being
 Photo credit: big data mind mapping,info graphics / Shutterstock

Education: The best protection against an economic crisis

by Dirk Van Damme
Head of the Innovation and Measuring Progress division, Directorate for Education and Skills

The insight that education is valuable both to individuals and to countries is not new. Using continuously improving data and statistical tools, we have come to understand and appreciate the magnitude of education’s impact on employment, income, health and life opportunities in general. From a purely economic point of view, private returns on investment are well beyond 10% per year, and public returns are only slightly below that figure. Fears that increasing participation and greater numbers of graduates – resulting in ever-increasing numbers of highly qualified people in the work force – would result in some kind of inflation, in diminishing returns and burgeoning graduate unemployment could not be confirmed by the data.

When the financial crisis erupted in 2007-08, rapidly turning into a global economic recession and a fiscal crisis in the Euro-zone and other countries, it was very difficult to predict its impact on education. Data for the years 2008 and 2009 showed that in the first years of the crisis, the impact on education remained limited and was confined to countries in severe crisis, such as Ireland, Iceland and Greece. Education is generally protected from shocks to the economic system because of its intrinsic slow pace of change. Individuals and families did not drastically alter their patterns of participation in education; and in the first years of the crisis, governments used stimulus packages and deficit spending to try to soften the blow, leaving education budgets more or less untouched. But we know that things started to change dramatically from 2010 onwards, when unemployment – especially among youth – climbed steeply and governments turned into austerity mode.

Education at a Glance 2013, the OECD’s reference on education indicators and statistics, for the first time provides a comprehensive set of data covering the years 2010 and 2011. The data convey a consistent, but also somewhat surprising picture. A great deal of the economic and social hardship caused by the crisis fell chiefly on less-educated individuals. The unemployment gap between well-educated young people and those who left school early widened during the crisis. On average across OECD countries, 5% of those with a tertiary education were unemployed, against 13% of those without an upper secondary education. Between 2008 and 2011 the unemployment rate for the latter group increased by 4%, while it rose by only 1.5% among the highly educated. The earnings gap widened as well: the average difference in earnings between highly educated and low-educated individuals grew from 75 percentage points to 90 percentage points between 2008 and 2011.

The message is clear: it is a person’s education that determines whether he or she will be extremely or only moderately exposed to the economic and social risks in times of crisis. Those without a minimal level of education, and certainly those of them without a stable job, find themselves without any shelter from the storm. Meanwhile, the relative returns on higher education increase. Some might argue that this is largely due to structural changes in the labour market: highly educated people taking the jobs of the middle-educated individuals, who, in turn, drive low-educated workers into unemployment. The earnings data do not seem to confirm this hypothesis: if tertiary-educated individuals were to take medium-skilled jobs en masse, their relative wage premium would not have increased as much as it did. In addition, the wage premium increases with age, which suggests that the higher level of skills among tertiary-educated workers is compensated in their salaries. In times of crisis and tough competition in the labour market, employers would certainly not be willing to value skills as much as they seem to do if there was no good reason to.

In short, the value of education increases during an economic crisis. Those who lack education, stand to lose a lot; those who have invested in it, can still reap its benefits.

Links:
For more information, and to download a copy of the book, visit the Education at a Glance website at: www.oecd.org/edu/eag.htm

Browse and share the book

For immigrant students, early arrival is best

by Marilyn Achiron
Editor, Directorate for Education and Skills

Arriving in a new country, in a new school as an immigrant student is never easy. But the transition can be a little less damaging if the student has already spent a few of his or her earliest years in his new home country. This month’s PISA in Focus examines the “late-arrival” penalty in student performance among immigrant students who arrived in their new country at the age of 12 or older.

An analysis of PISA data shows that there are no marked differences in reading proficiency between immigrant students who arrived in their new country before they were five and those who arrived between the ages of six and 11. In contrast, in most OECD countries, immigrant students who arrived at the age of 12 or older – and have spent at most four years in their new country – lag farther behind students in the same grade in reading proficiency than immigrants who arrived at younger ages. 

Countries and economies vary markedly in the magnitude of this “late-arrival penalty” for immigrant students. Differences tend to reflect the composition of the immigrant populations. Australia, for instance, has a large proportion of immigrants from the United Kingdom who already speak the same language as non-immigrant Australians. As a result, the average late-arrival penalty for immigrants in Australia is smaller than that in Germany, for example, where the largest groups of students who were born abroad come from the former USSR, the former Yugoslavia and Turkey.

An examination of age-at-arrival profiles for the major immigrant groups in selected countries confirms the importance of language barriers. In Luxembourg, French children do not suffer a late-arrival penalty; and age-at-arrival seems to make no difference to the reading performance among German students who immigrated to Switzerland. In contrast, 15-year-old students from the former Yugoslavia or Portugal who arrived in Switzerland or Luxembourg within the previous few years fare much worse in reading than immigrant students from the same countries who had spent all their school years in their new country.

But language may not be the only factor involved. Differences in educational and living standards between the origin and destination countries may also be relevant. Overall, an analysis of PISA data finds that immigrant students are particularly at risk of suffering a late-arrival penalty if they arrived at lower secondary-school age from less-developed countries where the home language is not the same as their new language of instruction. These students have to quickly acquire language skills and catch up with the higher levels of attainment achieved by their peers – while simultaneously coping with the difficulties of adjusting to a new school and social environment.

These findings can be used to inform immigration policy: Where late arrival is the result of migration policies that delay family reunification, the intended benefits of these policies should be carefully weighed against the costs of remedial assistance. More immediately, though, targeted help with language skills for those foreign-born students who arrive when they’re in their teens can limit the need for future assistance;and flexible arrangements to defer tracking can help to ensure that students perform at their full potential when decisions are taken about further education. Both measures will have a direct impact on these students’ employment prospects later on.

Links:   
For more information on PISA: www.oecd.org/pisa/

PISA in Focus No. 29: Do immigrant students’ reading skills depend on how longthey have been in their new country?

Photo credit: Children in school, from kindergarten, preschool, elementary / Shutterstock

Diverging levels of research efficiency change the global landscape of innovation

by Dirk Van Damme
Head of the Innovation and Measuring Progress division, Directorate for Education and Skills

Recovering from the current recession and building a sustainable economy based on growth and social progress require investments in innovative research. No future is imaginable without a new wave of innovation. But innovative ideas come from creative research, in many cases produced by universities and research institutions. Over the past decades countries have generally increased their investment in R&D, leading to higher numbers of researchers, more research activities and higher research output. Thanks to more sophisticated indicators and monitoring tools, we are now able to measure progress in research and to evaluate the impact of research in the global research system.

This global research system still is a rather concentrated system, with a very strong core and a wide periphery. Despite globalization and increasing access to research findings thanks to technology, many countries still are completely absent from this global community. Of the research universities included in the top 200 of the Times Higher Education World University Rankings 2012, more than half are in the United States (76) or the United Kingdom (31). In research the world is becoming flat only at a very, very slow pace. This is strange, because sharing ideas and knowledge on the Internet is even easier than transferring money. And, in contrast to money, sharing knowledge enriches both parties in the communication chain.

Yet, the massive size of the research infrastructure in the core countries is partly misleading. We know a lot about many input measures, but they tell us only one side of the story. What matters more is the impact of research. Bibliometric measures such as citation and impact scores, calculated from big databases, now allow seeing where the most relevant research is located. Next to research measures, since 2010 the new methodology of the Times Higher Education World University Rankings now also include good measures of research impact, with citation indicators developed with Thomson Reuters, the global leader in research database management. These data are now mostly used to compare universities, but we also can use them to compare countries and shifts in the global research system over time.

The United States still are world leaders when it comes to investments in research. The output of their universities in terms of size and impact puts the country among a select few in the world. Based on the average of their 76 universities in the 2012 ranking, the US score 85.21 points on the citations indicator. But the average score of French (83.40), Irish (81.50) and Swiss (81.20) universities in the top 200 follow closely. The strange thing is that the citations score does seem to have almost no relationship with the research score, which basically is a measure of input. Comparing individual universities on both measures, there is an extreme variation between the two scores. High investments do not at all generate automatically high outputs in terms of citations and impact. On a country level we even see a negative correlation between the research input score and the citations score of -.53. This is mainly due to Asian countries such as China, Chinese Taipei, Japan, Hong Kong, Korea, and Singapore, who make huge investments in research but still fail to generate impact. The average research input score of all these Asian countries is higher than the average score for US universities, while their average citations score is much lower. These countries learn that it doesn’t suffice to create an excellent research infrastructure, but that it takes time and probably specific policies to generate research which finds its way into the heart of the global knowledge system.

Dividing citations scores by research scores gives a proxy for research efficiency in a country, at least among the universities who can compete with the very best in the world. Countries with the best level of research efficiency are France, Germany, Denmark, Ireland, and Switzerland, and all of them do better than the US or the UK. Adding a time dimension makes the picture even more contrasting: these five countries are all improving their research efficiency over a period of two years with comparable data (2010-12), while China, Chinese Taipei, Korea, and Singapore, together with Australia and Austria are below average and decreasing in research efficiency (see graph above).

The global research system definitely is diversifying beyond the two traditional core countries with world-class universities, but the countries in the immediate periphery, mainly in Europe, are the ones who benefit from that. They are not as massively investing as for example Asian countries, but they do it much more effectively, probably by targeting their knowledge strategies better. Ultimately, it will be the research that matters which will find its way into innovation. Smart and targeted policies are probably much more effective than ambitious, but very broad investment policies.

Links:
OECD finds greater efficiency outside the elite: Times Higher Education
Centre for Educational Research and Innovation (CERI)
Chart source: OECD