Inteval

INTEVAL Notes

[AMITEL] Cédric — 2011-02-14T13:49:25Z

I. Introductory text

INTEVAL Evaluation Notes are short essays which address current issues in evaluation theory and practice. Some of the ongoing work by the INTEVAL group is discussed in the Notes series. In addition, we welcome original contributions from authors working in the field of evaluation (policy evaluation, program evaluation, (performance) auditing).
Potential authors who wish to contribute can contact John Mayne ( john.mayne@rogers.com ) or Jos Vaessen ( jlvaessen@gmail.com ).

II. Inteval guidelines for authors

III. Inteval notes

2011-1 : Independence in Evaluation and the Role of Culture – John Mayne

Performance Auditing: Contributing to Accountability in Democratic Government

Markus Spinatsch — 2010-10-26T09:22:00Z

Performance auditing:contributing to accountability in democratic government

Editors: Jeremy Lonsdale, Peter Wilkins and Tom Ling

Publishers: Edward Elgar Publishing, UK

How governments use their resources is a subject of considerable importance at a time when many countries are suffering economic pressures and citizens are seeking both lower taxes and better public services. Assessing in an independent manner how public funds are spent has never been more important. The role of performance audit has grown in many democracies as auditors have widened their view from whether the money has been spent as intended, to broader considerations of whether it has been spent efficiently and, even more challenging, whether it has been effective. At the same time, services have been delivered through longer and more complicated delivery systems and the intended benefits may be dispersed and hard to measure. In response, performance audit has sought to become more sophisticated in the methods used while still focusing on the central issue of value for money.

This book will examine the continuous development of performance audit in a number of countries around the world, including the United States, Australia, the United Kingdom, The Netherlands, and Belgium. It will start by sketching out the main directions in which performance audit has been moving, before considering what performance audit is contributing both to helping governments improve their services and in helping hold governments to account for their performance. It will look in detail at how auditors carry out their work – their use of evidence and methods, the selection of topics, the broadening of ways of communicating their findings - and consider the public policy implications of recent trends.

The book is an edited collection, produced by authors who have worked in or around audit offices for many years, and have written about them. In most cases, they have done both. Thus, the book has the perspective of both practitioners conversant with the academic literature on audit, and academics who have been involved in the practice of audit. The intention is to write the book in a way that is also of interest to those who are the subject of performance audits – officials in public sector bodies around the world and others who use the reports for accountability and performance improvement purposes.

Provisional chapter listing

Foreword (Paul Posner)
1. Introduction. Performance audit: An effective force in difficult times ? (Jeremy Lonsdale)
2. Performance audit: audit or misnomer ? (Jan-Eric Furubo)

The conduct of performance audit
3. Selection of topics (Vital Put and Rudi Turksema)
4. Norms used: some strategic considerations from Netherlands and the UK (Vital Put)
5. The Right Tools for the Job ? Methods and Context (Jeremy Lonsdale)
6. Evidence and argument (Alex Scharaschkin)
7. How do VFM audit teams know what they know (Justin Keen)
8. Quality and standards (Peter Wilkins and Richard Boyle)

The contribution of performance audit
9. What the critics say…..on effects, lack of effects and perverse effects (Frans Leeuw)
10. Accountability, performance and performance auditing: reconciling the views of scholars and auditors (Mark Funkhouser)
11. Impact– “slow and subtle” (Eddy van Loocke and Vital Put)
12. Impact at local government level: a multiple case study (Katrien Weets)
13. Learning in an accountability setting (Jeremy Lonsdale and Elena Bechberger)
14. Responsive performance audit (Peter van der Knaap)

Conclusion
15. Conclusion (Tom Ling, Peter Wilkins and Jeremy Lonsdale)

Current status: the team aim to submit the manuscript to the publishers, Edward Elgar Publishing in the summer of 2010.

Further information can be obtained from Jeremy Lonsdale at mailto:jeremy.lonsdale@nao.gsi.gov.uk

Turksema Rudi

[Eureval] joelle — 2010-06-24T07:12:12Z

Performance Audit Expert Netherlands Court of Audit Lange Voorhout 8 2514 ED The Hague The Netherlands Email : r.turksema@rekenkamer.nl phone +31703424297

Dr. Rudi Turksema is a Performance Audit Expert at the Netherlands Court of Audit. His work focusses on the quality and effectiveness of performance audits. This also involves the quality of performance information, evaluation studies, and results-based budgeting. Before coming to the Netherlands Court of Audit, he received his PhD in Sociology (on a study into the effect of government policy on day-care supply) from Utrecht University, The Netherlands.

Bohni Nielsen Steffen

[Eureval] joelle — 2010-06-15T08:42:07Z

Director of Evaluative Knowledge
Ramboll
Nørregade 7A
DK-1165 Copenhagen K
DENMARK
e-mail
Phone : +4529488103

Steffen Bohni Nielsen holds a PhD from University of Aarhus, Denmark. He is currently Director of Evaluative Knowledge at Rambøll Management Consulting where he is responsible for the knowledge and quality management of evaluation services delivered by 140 colleagues across Europe. For the past ten years he has planned, conducted and reviewed numerous evaluation studies within the human services in Denmark and abroad. He has also worked extensively with evaluation capacity building and results-based management. He has published extensively on these topics and serves as a referee for three international public management and evaluation journals. He is a former board member of the Danish Evaluation Society.

EVALUATION : SEEKING TRUTH OR POWER ?

[Eureval] joelle — 2010-06-01T13:01:03Z

to be published

ISBN : 978-1-4128-1141-5 Pages : 352

THE EVIDENCE BOOK

[Eureval] joelle — 2010-06-01T12:47:43Z

Knowledge grows as ideas are tested against each other. Agreement is not resolved simply by naming concepts but in the dialectical process of thesis, antithesis and synthesis. There are many echoes of these debates in The Evidence Book. The contributors make claims for both practitioner wisdom and the voice of experience. Against this is posed the authority of experimental science and the randomized controlled trial.

The contributors are concerned, in their own ways, with collecting, ranking, and analyzing evidence and using this to deliver evaluations. As an expert group, they are aware that the concept of evidence has been increasingly important in the last decade. As with other concepts it too often escapes precise defi nition. Despite this, the growing importance of evidence has been advocated with enthusiasm by supporters who see it as a way of increasing the eff ectiveness and quality of decisions and of professional life.

The willingness to engage in evidence-based policy and the means to do so is heavily constrained by economic, political and cultural climates. This book is a marvelously comprehensive and utterly unique treatise on evidence based policy. It is a wide-ranging contribution to the fi eld of evaluation.

Olaf Rieper is Director of Research at the Danish Institute of Governmental Research (AKF). He has written books and guidelines on evaluation methodology and research methods and was the first elected chairman of the Danish Evaluation Society.

Frans L. Leeuw is the Director of the Justice Research, Statistics and Information Center, affi liated to the Dutch Ministry of Justice and Professor of Law, Public Policy and Social Science Research at Maastricht University. He is one of the founders of the European Evaluation Society.

Tom Ling is Director for Evaluation and Audit at RAND Europe. He is Professor of Public Policy (Emeritus) at Anglia Ruskin University where he contributes to postgraduate teaching.

ISBN : 978-1-4128-1023-4 Pages : 191

MIND THE GAP: PERSPECTIVES ON POLICY EVALUATION AND THE SOCIAL SCIENCES

[Eureval] joelle — 2010-06-01T12:36:49Z

Over the past twenty to thirty years, evaluation has become increasingly important to the field of public policy. The number of people involved and specializing in evaluation has also increased markedly. Evidence of this trend can be found in the International Atlas of Evaluation, the establishment of new journals and evaluation societies, and the increase in systems of evaluation. Increasingly, the main reference point has become an assessment of the merit and value of interventions as such rather than the evaluator's disciplinary background. This growing importance of evaluation as an activity has also led to an increasing demand for the type of competencies evaluators should have. Evaluation began as a niche area within the social and behavioral sciences. It subsequently became linked to policy research and analysis, and has, more recently, become trans-disciplinary. This volume demonstrates an association between the evaluation tradition in a particular country or policy field and the nature of the relationship between social and behavioral science research and evaluative practice. This book seeks to offer comprehensive data, which lead to conclusions about patterns that transcend the gap between evaluation and the social scientific disciplines. Mind the Gap has a twofold aim. The first is to highlight and characterize the gap between evaluation practices and debates, and the substantive knowledge debates within the social and behavioral sciences. The second is to show why this gap is problematic for the practice of evaluation, while at the same time illustrating possible ways to build bridges. The book is centered on the value of producing useful evaluations grounded in social science theory and research.

0. Introduction: mind the gap (Frans L. Leeuw and Jos Vaessen)

PART I: The Evolving Relationship between Evaluation and the Disciplines

1. US sociology and evaluation: issues in the relationship between methodology and theory (Nicoletta Stame)
2. Evaluation and the disciplines in development (Robert Picciotto)
3. Economics and evaluation (Sandra Speer)

PART II: Evaluation and its Disciplinary Basis

4. The intellectual underpinnings of a comprehensive body of evaluative knowledge: the case of INTEVAL (Lisa Birch and Steve Jacob)
5. Public management theory, evaluation and evidence-based policy (Robert Schwartz)

PART III: Bridging the Gap between Evaluation and the Disciplines

6. Interventions as theories: closing the gap between evaluation and the disciplines? (Jos Vaessen and Frans L. Leeuw)
7. Middle-range theory and program theory evaluation: from provenance to practice (Ray Pawson)
8. Realistic evaluation and disciplinary knowledge: applications from the field of criminology (Nick Tilley)

ISBN: 978-1-4128-1075-3 Pages: 254

Nouvel article

[Eureval] joelle — 2010-05-31T14:10:27Z

- Forum

Discussing cutting-edge issues

[Eureval] joelle — 2010-05-31T09:57:13Z

In this page, INTEVAL group members engage into a series of public discussions about new approaches to (and new viewpoints on) policy and program evaluation. Just register and start interacting. A first pilot discussion has opened on contribution analysis.

Contribution analysis is a theory-based approach to impact evaluation. It proceeds through a step-by-step investigation into all cause-and-effect assumptions. It does not require developing a counterfactual. It enables evaluators to reach robust conclusions on the impact of public interventions even where they involve complex and open arrays of causal assumptions, for instance in the case of prevention of health problems or improvement of governance.

This discussion is moderated by Jacques Toulemonde and by John Mayne who introduced the concept of ‘contribution analysis' in 1999.

Knowing more about contribution analysis:
•a brief introduction to the approach
•a presentation at a recent conference of the UK Evaluation

Evaluation and cultures

[Eureval] joelle — 2010-05-28T07:49:09Z

Cultures do affect the conduct, outputs, and utilisation of evaluations. The book reflects on how evaluation cultures take shape in the context of wider cultural environments (political, administrative, professional, organizational, …) at the level of countries or institutions.

The book is in progress. Its provisional contents are:

Evaluation and Culture, a Multidimensional Relationship - Jean-Claude Barbier and Penny Hawkins
Independence in Evaluation and the Role of Culture - John Mayne
Evaluation Cultures: Singular or Plural? – Kim Forss
The culture of evaluation in Italy - Nicoletta Stame
Riding the Celtic Tiger: forces shaping evaluation culture in Ireland in good times and bad - Richard Boyle, Gery Mc Namara and Joe O'Hara
Exploring the Culture(s) of Evaluation in Swiss Health Care - Markus Spinatsch
Cultures of Evaluative Activity in the Provision of Acute Healthcare - Andrew Gray and Barbara Coyle
Sectoral Evaluation Cultures: A Comparison of the Education and Labor Market Sectors in Germany - Sandra Speer
Evaluation Cultures in the EU: learning from similarities and differences across policy areas - Jacques Toulemonde
The Culture of Evaluative in Switzerland: a Reflection of its Multiculturalism? - Katia Horber-Papazian and Caroline Jacot-Descombes
... there are stated intentions for four other chapters
Conclusion