Evaluationskompetenz

Date:    Wed, 21 Jul 2004 08:29:49 +0100
From:    bill fear 
Subject: Agendas, decisions and using evaluation


The debate about the role of the evaluator in relation to getting the
evaluation used has had a long and perennial history re-emerging, as ever,
about once every five years.  There are a couple, or more, important points
that are consistent (IMESHO):


1)  No evaluator has the right to assume that their findings will, or
should, be used as a number of people have just recently noted.  This right
is the preserve of auditors.
2)  There are two ways to maximise the value of an evaluation:  a) involve
stakeholders from the off (Patton); b) link evaluation to budgets
(Australia; the Netherlands).
3)  Most interestingly, a piece of work by by the NAO (probably by Chelimsky
and published around 5-8 years ago; sorry, I have a real problem remembering
references) showed that high quality evaluations tend to be rejected
initially.  However, these same evaluations usually have an impact around
five years - that's 5 years - later, usually at the conceptual level.  Ergo,
an acid test of a good evaluation that has been carried out independently of
stakeholders may well be the degree of initial resistance and rejection.
Indeed, it may be that an evaluation has more impact if the evaluator does
not try to get it taken account of.  Just think through what we know about
decision making.


On that point, any good evaluator surely must, surely absolutely must, have
an understanding of decision making from the individual level to the
organisational level.


Helpful references are:


at an individual level


www.bps.org.uk then click on 'publications' then 'the psychologist' then
'search the psychologist online' then 'volume 15 (2002)' then 'volume 15
part 2(February 2002)' then look at articles 4, 5, 6, 7.  Easy reading to a
high standard (mostly).


and


Gilbert, D. and Wilson, T.  'Miswanting.'
www.wjh.harvard.edu/~dtg/Gilbert%20&%20Wilson%20(Miswanting).pdf  (or put
'miswanting' into google)


At an organisational level it is still, for me, the stock in trade
publication of 'Organsiations:  Structures, processes and outcomes' by Hall.


We might also want to consider that US Senators apparently spends just 7
minutes a day reading on average and that for a GP to keep up to date with
current relevant medicine they need to read for 17 hours a week (mostly
non-fiction, or at least not knowingly fiction).


And then of course there is the values of the evaluator.  Our values tend to
drive our behaviour - although they don't have to.  Not judging others on
the basis of their values, which may conflict heavily with our own, is
immensely difficult.  So, we may assume that our evaluation should be taken
account of according to our values, but the values of the person on the
other side may be different.  And somehow we have to find a way not to let
that influence our behaviour and to respect the values of the other/s.
After all, there is no moral 'right' or 'wrong', and ethics are consensus of
agreed rules depicting right and wrong, and not a universal absolute, and
there is no known set of universal values.