Year: 2002
Type: editorial
Abstract: Evaluators work with data and evidence, with concepts and theories and with users and stakeholders.All these aspects of evaluation are represented in this issue of Evaluation.Pawson, Grocott and her colleagues and Vartiainen are all concerned in their different ways with comparing, combining and generalizing from evaluation data and evidence.Patricia Grocott, Sarah Cowley and Alison Richardson confront the methodological difficulties of measuring in order to control the symptoms of patients with severe 'malignant' wounds.In the medical world where experimental methods dominate, the authors have nonetheless found it difficult to apply a quasi-experimental design.According to Grocott and her colleagues, because these wounds and their extent could not be measured and because individual experience of apparently similar distressing symptoms varied so greatly, quasi-experimental methods failed.They failed because 'the assumptions about the nature of objective knowledge' and how this could be accessed, 'were inaccurate'.The authors instead adopted a theory-driven, constructivist approach, using case-study methods in which each individual person and their wounds were regarded as a single case.Abstracting general knowledge followed a 'quasi-judicial' approach to build up 'case law' by consensus.Pirkko Vartiainen directly addresses what she calls the 'principles' of comparative evaluation.Her material for comparison is organizational data -but her argument is far broader.Vartiainen takes forward an argument that has featured in this journal before, perhaps most notably in Cheetham, Mullen, Soydon and Tengvald in issue 4.1.Vartiainen argues that how one calibrates her four principles: 'the selection of the object'; 'the level of comparison'; 'conceptual comprehension' and 'analysis of findings' makes it more or less possible to obtain comparable and reliable information.Yet all of these principles, the author suggests, rest on how well concepts, indicators and criteria are specified, as well as the innate similarities of what is being compared.Ray Pawson takes forward the arguments in favour of a 'realist' basis for systematic reviews as part of 'the quest for evidence-based policy'.Whereas Pawson's article in the last issue of Evaluation critiqued existing evidence-based review approaches, this article outlines the case for an alternative.Realist synthesis, Pawson argues, differs in terms of its approach to causation, its ontology and its approach to generalization.Following this realist logic, it is more sensible to consider 'families of mechanisms' rather than 'families of programmes'.After all, similar mechanisms may apply across many programmes.As those familiar with Pawson's work would expect, he is less concerned with adding up success and failure to assess evidence.Rather he assumes all mechanisms may be both successful or not -depending on context.Rita O'Sullivan and Anne D'Agostino argue that adopting a collaborative stance towards programme staff and stakeholders 'will strengthen evaluation results' and make it more likely that 'evaluation findings will be used for programme improvement'.To this end the authors conducted an empirical study in the context of community-based programmes for children and their families in North Carolina.The authors applied a type of collaborative evaluation, they label 'Evaluation Voices' -a variant of 'cluster' evaluation combined Evaluation
Source: Evaluation
Author: Elliot Stern
Institution: Tavistock Institute
Language: Spanish
Subfield: Public Administration
Field: Social Sciences
Domain: Social Sciences
Open Access status: bronze
OpenAlex ID: W4256393886
