Balancing Trust and Accountability

by Tracey Burns
Analyst and Project Leader, Innovation and Measuring Progress Division, Directorate for Education and Skills

Strengthening accountability is one of the key ways to improve the quality of an education system. Yet reform processes that emphasize strong evaluation and assessment regimes can be misunderstood as controlling or demonstrating a lack of trust: in teachers, in students, and in the system. What is the best way to maintain and build trust while improving accountability?

A recently released Governing Complex Education Systems (GCES) case study looks at this issue. Entitled Balancing Trust and Accountability? The Assessment for Learning Programme in Norway, the report explores the implementation strategies used to enhance formative assessment in Norwegian schools. The reform aimed at helping school leaders and teachers integrate formative assessment into their day-to- day teaching practice and schools.

In Norway there is a strong sense of trust in the system. However, when the system relies wholly on trust and has few incentives (or sanctions) for the actors, long-term implementation of policy initiatives can become problematic in the face of resistance.

In the last several years Norway has been one of many OECD countries working to build a culture of assessment across the entire system. In order to achieve this, governments have to strike the right balance between providing clear directions and goals on the one hand, and allowing for freedom of innovation and creativity in practice on the other. This is true for teachers and other stakeholders as well: trust in local authorities, for example, is also required for the effective functioning of the system.

Effective formative assessment, like teaching, is based on a combination of factors and there is no single recipe for success. However in order to learn new practices, teachers are just like the rest of us – they need guidance and support. In this case, the Norwegian authorities have provided workshops, online tools, and peer learning networks to help facilitate the process. But it is not easy – despite the support put in place, the majority of the teachers interviewed for the case study struggled with understanding what would be considered “correct” practice.

So what can be done? Prepare the ground. Don’t underestimate the importance of clear goals and communication. And recognize the power of networks. In this case study, the research team found that successful implementation of the programme was enhanced by:

• strong leadership skills and thorough knowledge of the content of the programme on the part of the municipalities;
• clear communication between governance levels and a high degree of trust between stakeholders;
•  a solid understanding of the programme goals, including integrating these goals within the broader aims of educational policy and school practice; 
• the establishment of learning networks between schools to aid the exchange of knowledge and opportunities for peer learning. 

One final note: This case study addresses the process of reform and the dynamics of change in a large-scale implementation of policy. In this context, the importance of strategic planning and alignment of goals between governance levels cannot be understated. There is a danger of too much, too soon – smaller municipalities in particular struggle with a continual stream of policy changes and prioritising activities. Meaningful change takes time, and embedding new behaviours into practice requires thoughtful and deliberate planning and feedback.

Education systems have never been easy to manage. Today it is abundantly clear that processes of reform cannot be understood as top-down unilateral delivery chains and treated as systems that engineer processes. Rather, they require reappraisal, fine-tuning, responsiveness and sometimes new structures of collaboration, participation and networking. And above all, trust must be nurtured, for it is hard to gain and easy to lose. As the old adage says, trust “arrives on foot and leaves on horseback”.

Links: 
OECD'S Centre for Educational Research and Innovation (CERI)
Governing Complex Education Systems (GCES)
OECD Review of Evaluation and Assessment for Improving School Outcomes
OECD Working Paper: Stakeholders and Multiple School Accountability
Photo credit:Symbol Scales is made of stones of various shapes/@ Shutterstock

Big data and PISA

by Andreas Schleicher
Deputy Director and Special Advisor on Education Policy to the OECD's Secretary-General

Big data is the foundation on which education can reinvent its business model and build the coalition of governments, businesses, and social entrepreneurs that can bring together the evidence, innovation and resources to make lifelong learning a reality for all. So the next educational superpower might be the one that can combine the hierarchy of institutions with the power of collaborative information flows and social networks. More than anything else, this will hinge on getting people to generate innovative applications on top of big data. It’s about the co-creation of governance, about delivering more progressive and better policies than the industrial work organisation and the bureaucratic and litigation-oriented tools and strategies that we are used to in education.

This isn’t just about improved transparency and public accountability in education. Throwing education data into the public space does not change the ways in which students learn, teachers teach and schools operate. It does not lead to people doing anything with that data and transforming education in ways that will actually change education practice. On the contrary, it often results simply in adversarial relationships between civil society and government over the control and ownership of information.

The prerequisite for using big data as a catalyst to change education practice is to get out of the “read-only” mode of our societies. It’s about combining transparency with collaboration. The way in which educational institutions often work is that you have a single expert sitting somewhere in a corner who determines the application of rules and regulations affecting hundreds of thousands of students and teachers – and nobody can figure out how those decisions were made. Big data can lead to big trust if we make that data available, train civic innovators, experiment, create a maker culture. It is no surprise that OECD’s new Survey of Adult Skills shows that the more proficient people are in literacy, the more they trust others.

Collaborative consumption provides a great example of this. These days, people share their cars and even their apartments with strangers. Collaborative consumption has made people micro-entrepreneurs – and its driving engine is building trust between strangers. Think about it: in the business world, we have evolved from trusting people to provide information, to willingly handing over credit card data, to connecting trustworthy strangers in all sorts of marketplaces. We are light-years away from that when it comes to data about education.

But here’s how we can get a little closer. Some years ago we created PISA, a global survey that examines the skills of 15-year-olds in ways that are comparable across countries. PISA has created huge amounts of big data about the quality of schooling outcomes. PISA has also helped to change the balance of power in education by making public policy in the field of education more transparent and more efficient. At the micro-level, there were still a lot of sceptics: teachers thought this was just another accountability tool through which governments wanted to control them. So what did we do? This year we put in place a kind of “MyPISA” – PISA-type instruments that we circulated out into the field. The PISA based test for schools provides comparisons with other schools anywhere else in the world, schools that are similar to them or schools that are very different.

Suddenly, the dynamic has changed; schools are beginning to use that data. Ten schools in Fairfax county in Virginia, for example, have started a year-long discussion among principals and teachers based on the results of the first reports. With the help of district offices (and the OECD), they will be conducting secondary analyses to dig deeper into their data and understand how their schools compare with each other and with other schools around the world. Those principals and teachers are beginning to see themselves as teammates – not just spectators – on a global playing field. In other words, in Fairfax county, big data is building big trust.

Links:   
For more information on PISA:www.oecd.org/pisa/
OECD Survey of Adult Skills (PIAAC)
The Role of Data in Promoting Growth and Well-Being
 Photo credit: big data mind mapping,info graphics / Shutterstock

The language of dissent

Language matters to Lea Rosenberg.

The former English teacher and high school principal who is now Deputy Director General of International Relations and UNESCO at Israel’s Ministry of Education recently made an impassioned appeal to fellow education policy makers that they, essentially, watch their language when talking about education. Using the terminology of other disciplines to discuss education policy, Rosenberg argues, does a disservice to education. Rosenberg was speaking at a high-level meeting of representatives of education ministries held at the OECD’s Paris headquarters in mid-October.
Marilyn Achiron, Editor at the OECD’s Directorate for Education, spoke with her after the meeting.

Lea Rosenberg: Throughout the years, when different paradigms from different worlds were introduced into the world of education, people were fascinated by the opportunities that these paradigms gave to education. They enabled larger and deeper views into what education is all about. At the same time, the status of education, compared to other professions, is usually low. So take these two things together and you arrive at where we are now.

We have talked about education through the sociological paradigm, where students are not people, but parts of populations. Of course, teachers should take note of the populations to which students belong and understand them, but that should not be the leading language of education. It is a “foreign language”. Understanding how students and teachers function is fine, but when we used the paradigm of psychology to discuss education – another “foreign language” – teachers started understanding students better, but they lost the notion of demanding something from students. If you don’t demand something from students, you cripple them; you treat them as though they are incapable.

Now we are using management language – talking of “inputs” and “outputs” – and we’re disappointed that the results are not what we expect. Principals and teachers should know and understand this language, but it is not education’s work. The same is true with the language of economics. We depend on economics: it is crucial, elementary to our existence. But this is not the language that should lead the world of education. For example, the word “accountability”: it deals with accounts. I would prefer that our teachers are “responsible” and not only “accountable”. Responsibility covers the range of human emotions, behaviours and activities – elements, such as caring, that are not covered by “accountability”. Students might do well in mathematics, but their morals and values might be terrible.

That is what we should be concerned about: What kind of person comes out of our education systems? The language of education focuses on being responsible for students as whole beings. Of course achievement is important, but students are human beings. The language of education is one of helping students overcome difficulties, taking differences into account, steering students through their mistakes so that they learn something and can then move on. Terminology affects the way we see the world. It affects the way we see students and the way we act towards them. This is especially true in education, where language is the only tool we have.

Links: 
Languages in a Global World: Learning for Better Cultural Understanding
Preparing Teachers and Developing School Leaders for the 21st Century - Lessons from around the world
Photo credit: Dictionary entry for "education"/Shutterstock

The more the merrier

by Tracey Burns
Analyst, Innovation and Measuring Progress Division, Directorate for Education

Who is responsible for successes and failures of schools? A new Education Working Paper says  involving parents and students can help improve education systems by including them in accountability and school achievement processes. The traditional approach where central government provides the resources and made the majority of decisions has given way in many OECD countries to greater autonomy and control over decision-making for schools and local governments. This greater freedom has developed hand in hand with the rise of benchmarking and international assessments, and has made accountability a hot topic for policy makers and communities alike.

But what about other voices? Parents, community leaders, and others are taking an increasingly active role in governing their local schools. This trend, called “multiple accountability”, aims to provide more localized and nuanced feedback and guidance that schools and education systems can use in addition to standardized test results. It is a promising concept that builds on successes from other public policy areas such as environment and health. There is also fascinating evidence from the business world that suggests that enabling shareholders in private corporations to vote on the pay policy of the company’s executive officers appears to lead to large increases in the company’s market value, profitability and long-term performance. These “Say-on-Pay” regulations, promoted in the US and UK, allow more people within the corporation to have a say in decisions – and crucially, this shared responsibility can result in benefits to the whole company.

In education, multiple accountability is still a fairly new concept, and the amount of available research on how to make it work is modest. Three lessons, however, can be learned from existing models in The Netherlands and the United Kingdom:

1. You must identify the key stakeholders. This is more difficult than it sounds, and schools must make efforts to involve less powerful or inactive voices.
2. You must build capacity for this new role. Some stakeholders might not have the knowledge and language needed and may inadvertently be excluded in accountability processes. Providing them with the tools to interpret and analyse benchmarking data and other evaluation processes (e.g., value added measures) is an important part of giving them the expertise they need to take part.
3. You must be ready and open to assess your school’s quality and processes . School leaders play a key role in empowering staff to be involved and open to parents and members of the local community.  

Including the voices of parents and other stakeholders could be one of the most relevant shifts for education policy today. But unforeseen challenges may arise: it turns out that market mechanisms such as school competitiveness and parental choice in education can actually be disincentives for making multiple accountability work. In a competitive market, transparent discussion of the weaknesses of a school can threaten the image and competitiveness of the institution. In such contexts, it is strategically important to highlight the successes and avoid talking about room for improvement. The real question is: which approach is best-suited to improving our schools and education systems?


Links:
Education Working papers
OECD'S Centre for Educational Research and Innovation (CERI)
Governing Complex Education Systems (GCES)
Photo credit: Talk in colours speech bubbles /Shutterstock

“We do things differently here”: evaluation and assessment in New Zealand schools

by Deborah Nusche
Policy Analyst,  Early Childhood and Schools Division, Directorate for Education

New Zealand’s consistent high performance in the OECD’s Programme for International Student Assessment (PISA) has sparked international curiosity about the ingredients of its success.

New Zealand’s education system is unique in many ways. It has probably gone furthest among OECD countries in allowing schools to run themselves. In turn, it’s not surprising that evaluation and assessment is very much in the hands of schools and their Boards, and the main policy focus has been to build their capacity to do this. Notably, student assessment relies strongly on the professionalism of teachers to assess and report on student learning. A new OECD report on evaluation and assessment in New Zealand schools provides in-depth information about the country’s unique approach to evaluating student, school and system progress.

What struck the review team most about New Zealand’s approach was the great amount of trust in the ability of students, teachers and schools to evaluate their own performance and engage in self-improvement. While international developments are closely followed, the global trend towards high-stakes accountability is not seen as a good option for New Zealand. Especially in primary education, there is a general consensus against national testing and the use of test results for school rankings.

To gather information on how the education system is doing overall, New Zealand relies on sample-based surveys that do not carry high stakes for individual students, teachers or schools. Instead of going further down the road of national assessments, New Zealand is investing in teacher capacity and guidance materials to help teachers make and report professional judgments about the learning of each student. The national agencies provide clear performance expectations and a set of nationally validated assessment tools to guide assessment practice. Teacher professionalism is also supported by well-established approaches to teacher appraisal and school self review. Both promote evidence-based inquiry and the use of assessment results by schools for accountability and improvement.

The New Zealand model has successfully avoided some of the potential negative effects of high-stakes testing such as curriculum narrowing, teaching to the test and assessment anxiety. It has helped communicate the message that assessment is an integral part of everyday teaching and learning rather than a one-off event at the end of the school year. Effective assessment is described by the Ministry of Education as a circle of inquiry, decision-making and transformation – in short, “a process of learning, for learning”.

While New Zealand has a lot to be proud of, the OECD report also identifies a range of challenges and provides recommendations for improvement. Policy priorities are to:

• Further develop and embed the National Standards within the evaluation and assessment framework
• Consolidate teaching standards and strengthen teacher appraisal 
• Strengthen school collaboration and regionally-based support for schools
• Reinforce professional learning opportunities for teachers, school leaders and trustees
• Ensure that evaluation and assessment respond to diverse learner needs
• Enhance consistency of the overall evaluation and assessment framework

Links

OECD Reviews of Evaluation and Assessment in Education: New Zealand:

• Country Review  
• Main Conclusions 
• Policy Pointers

For more on OECD Reviews on Evaluation and Assessment Frameworks for Improving School Outcomes: www.oecd.org/edu/evaluationpolicy

Related blog posts:

When the school inspectors call less often, will Flemish schools take up the self-evaluation challenge?

Snowed under or crystal clear? Evaluating and assessing education outcomes in Norway

 The report was authored by Deborah Nusche, Dany Laveault, John MacBeath and Paulo Santiago


Photo credit: New Zealand Ministry of Education