|
p.20 In program evaluation, as in life, it is one's overall philosophy integrated into pragmatic principles that provides a guide to action.
p.22 Utilization-focused evaluation does not advocate any particular evaluation content, model, method, theory, or even use. Rather, it is a process for helping primary intended users select the most appropriate content, model, methods, theory, and uses for their particular situation. Situational responsiveness guides the interactive process between evaluator and primary intended users.
p.179 what gets measured gets done.
p.180 the degree of uncertainty facing an organization directly affects the degree to which goals and strategies for attaining goals can be made concrete and stable... Effective organizations in turbulent environments adapt their goals to changing demands and conditions.
p.189-190 Focusing an evaluation means dealing with several basic concerns. What is the purpose of the evaluation? How will the information be used? What will we know after the evaluation that we didn't know now? What actions will we be able to take based on evaluation findings? These are not simply rote questions answered once and then put aside. The utilization-focused evaluator keeps these questions front and center throughout the design process. The answer to these and related questions will determine everything else that happens in the evaluation.
p.190 Focusing involves a choice. The decision to look at something is also a decision not to look at something. A decision to see something means that something else will not be seen, at least not with the same acuity... The same principle applies to evaluation. Because of limited time and limited resources, it is never possible to look at everything in great depth. Decisions have to be made about what's worth looking at... Utilization-focused evaluation suggests that the criterion for making those choices of focus be the likely utility of the resulting information.
p.190-191 Making use the focus of evaluation decision making enhances the likelihood of, but does not guarantee, actual use. There are no guarantees. All one can really do is increase the likelihood of use.
p.199 The problem with pure outcomes evaluation is that the results give decision makers little information to guide action.
p.215 How are we to proceed without Theory? ...Change? Yes, we must change, only show me the Theory, and I will be at the barricades, show me the book of the next Beautiful Theory, and I will promise you these blind eyes will see again, just to read it, to devour that text. Show me the words that will reorder the world, or else keep silent. -Kushner 1994:13-14
p.216 When studying causality as a graduate student, I marveled at the multitude of mathematical and logical proofs necessary to demonstrate that the world is a complex place (e.g., Nagel 1961; Bunge 1959).
p.217 Evaluation offers reasonable estimations of probabilities and likelihood, enough to provide useful guidance in an uncertain world (Blalock 1964).
p.217 To venture into the arena of causality is to undertake the task of theory construction.
p.221 The phrase theories of action refers specifically to how to produce desired results in contrast to theories in general, which explains why some phenomenon of interest occurs.
p.222 what is perceived as real is real in its consequences.
p.223 The espoused theory of action is a straightforward articulation of what is supposed to happen in the process that is intended to achieve desired outcomes. The linkages between processes and outcomes are made explicit.
p.225 The purpose of thoroughly delineating a program's theory of action is to assist practitioners in making explicit their assumptions about the linkages between inputs, activities, immediate outputs, intermediate outcomes, and ultimate goals. Suchman (1967) called beliefs about cause-effect relationships the program's validity assumptions... In a utilization-focused evaluation, the evaluator works with the primary intended users to identify the critical validity assumptions where reduction of uncertainty about causal linkages could make the most difference.
p.226-227 Delineating an espoused theory of action involves identifying critical assumptions, conceptual gaps, and information gaps.
p.232 A theory of action is at least partially temporal in conceptualization because it progresses from immediate objectives to ultimate goals... Program components may be conceptually distinct in the formal version of a theory of action, but in practice these analytically distinct components, links, and stages are highly interdependent and dynamically interrelated. In short, the cause-effect relationships may be mutual, multidirectional, and multilateral.
p.237 While causal linkages may never be established with certainty, the delineation of assumed causal relationships in a chain of hierarchical objectives can be a useful exercise in the process of focusing an evaluation.
p.237 a theory can be the key that unlocks the door to effective action.
p.385 Each evaluation being a blend of unique ingredients, no standardized recipe can ensure the outcome. We have only principles, premises, and utilization-focused processes to guide us, and we have much yet to learn.
p.385 Build the study of use into your evaluations... Experiment with ways of making evaluation useful
|
