OAFP 8115 - Evaluation of Transition policies

Over the last 60 years, public policy evaluation has become increasingly widespread, first in North America and the Global North, and then globally. Part of this success is due to the peculiarities of the approach. Evaluation uniquely combines three features: 1) a judgement of the value of policies in terms of their consequences (rather than their intentions), 2) based on empirical evidence gathered using social sciences protocols and methods, 3) with a view to being helpful to policy stakeholders and to policy making in general. But evaluation is also characterised by strong tensions, both within the practice (e.g. between a social science ethos and judgement) and as part of the very policy it seeks to evaluate. This has led to many controversies (Is evaluation speaking truth to power or the truth of power? How useful is evaluation if policies are not significantly improved? Does evaluation ultimately contribute to the public good?), but also to a wide range of approaches, methods and stances that make contemporary evaluation practice very diverse and versatile.

This course provides students with an overview of the challenges and debates in contemporary evaluation at the intersection of politics, policy and the social sciences; it discusses attitudes, approaches and methods in evaluation and introduces the audience to evaluative thinking as a way of assessing and making sense of policy. It is set in the specific context of the current era of polycrises and socio-ecological transitions, an age of radical uncertainty in which evaluation has a role to play in making sense of wicked problems and clarifying what is known and what is not, but can also be a tool of the status quo. A very practical approach to these problems is taken as students follow the actual process of evaluating a transition policy throughout the course.

It is recommended to have followed basic courses in programme evaluation, in policy analysis, and on sustainable development.

The course gives an important role to groupwork and dialogue between the students and with the teacher. Readings and preparation between sessions are essential to foster a high-quality conversation.

• A collective grade (50%) — a collective work (subgroups) on a case study, in which students ask a causal question, consolidate a programme theory to respond to that question, and suggest a methodological approach to answer it.
• An individual grade (40%) — a 500 word note in which students formulate an alternative evaluation question (within their subgroup case study), and suggest a methodological approach to answer them
• A collective grade (10%)— from session 2 to 11, each on their turn, subgroups are asked to write a 500-word summary of what they have learned during the session, in a document that is open to all students, who can update or comment it. Participation in this collective work automatically grants maximum points for this grade.

Students work on real-life transition challenges, such as evaluating the positive and negative consequences of a low-emission zone, assessing the inclusiveness of a cycling programme, examining how ‘sustainability challenges’ can influence daily behaviours, investigating the impact of deep renovations on the energy consumption of poorer households, considering the expansion of organic waste recycling, or exploring the greening of manufacturing processes in European SMEs.

Students will be expected to prepare session 1 individually, and the other sessions in subgroups. In general, this might take an hour per week. Once or twice in the duration of the course, students will also be requested to write a summary of the session in <500 words.