A few months ago Joe Saxton, Driver of Ideas at nfpSynergy,
wrote a thought-provoking blog
challenging our sector’s enthusiasm for randomised control trials. Joe’s
article and the debate started in the comments, were fascinating and deserve a read.
But it prompted consideration of the question one step removed – before we look
at charities borrowing science’s tools, is it appropriate to compare charitable
and scientific worlds?
Randomised control trials are the current highest point of
evidence collection in the scientific method, a method whose genesis is
hundreds of years old and whose structure is supported by countless examples of
error, trial, error, improvement. When scientists operate, their experiments
rest not just on the shoulders of giants, but on the backs of a pyramid of
giants, trolls, charlatans and visionaries. Science learns from itself, from
its mistakes, around the globe and across the centuries.
The scientific method demands that results be replicable,
and expects an important experiment will be run by entirely different people
time and again. Science has operated for decades within the infrastructure of
the academic world, with a host of peer review journals and challenging
conventions, allowing distant practitioners to test the validity of claims and build
on success. Rivals and successors pore over datasets, read failed experiments
and negative results, perfect techniques. The world has far more STEM graduates
than experts in charitable operation or social policy research. The sector’s
main notable academic journal is that of the Wellcome Trust, Wellcome
Open Research. It is excellent, and it is, of course, a science journal.
This is not the only difference. Science and technology are
wedded more firmly to economic progress.
Shareholders, government, hospital directors
and the public take note of new drugs, stem cell breakthroughs, rumours of
groundbreaking green energy generators. Charity practitioners have nowhere near
this level of awareness – not because we are lazy or intellectually inferior,
but because we have no such support structure or history of sharing. Too often
in the charity sector, it is not just a case of one hand not knowing what the other
hand is doing: it is two fingers on the same hand each reaching out to grasp
the same object and still failing join up.
In the corporate world, certainly in the boardrooms of
pharmaceutical and tech companies, directors are inquisitive and acquisitive.
There are aware of all their competitors’ projects, what newcomers try, innovation
springing up in far-flung corners, blossoming SMEs. They are not only concerned
with keeping their own company afloat but with exploring expansion on the
frontier. They have teams of researchers comparing clusters of studies and
meta-analyses to scope opportunity. They are supported by both the academic
literature and by business media – the Financial
Times and rolling TV news. Likewise they have a worldwide network of business
schools, economics departments, management courses, decades of theories on
effective leadership and proactive governance.
There are of course a great number of collaboration efforts
in the charity sector, from the Good Exchange to the concept of “generous
leadership”, from joint initiatives between funder organisations and umbrella
bodies to local projects in the same town or village. One of Charity Futures’
ambitions is to compile a directory of these, listing free and paid resources
on charity academia, leadership and governance training, and emergency support.
Hopefully by signposting both collaborations and smaller ventures, even more
efficient partnerships can be forged. This could grow in utility by adding
neutral reviews and learning aids, so a bewildered new board member could
easily find out the different tools available to help her.
The other difference between science and voluntary worlds is
simply that a lot of charities do not operate in a manner with quantifiable
results. The goal of some is to enable a sport to be played, or to make a
group’s life more tolerable, hopefully enjoyable. There are sector activities
which suit social science measurements, like helping ex-prisoners reintegrate
or educating children, but a host of important charitable activities are little
to do with numbers. Has enough thought been devoted to testing an ethical
component, are quality-adjusted life years enough?
Trying to get a picture of impact by asking beneficiaries to
rate their experiences feels like missing the point, even if methodologies were
sound enough that they could be compared across location and type – which they
aren’t. Some of the largest management consultancies have been trying for years
to set out a standardised system to rate charity effectiveness and each model
sinks on its flaws. What the voluntary sector does brilliantly is use hard
science evidence in campaigning – against smoking near children for example –
and funds investigation of this type. But that does not contribute to a central
corpus on how charities themselves campaign.
The question of randomised control trials speaks to the
charity bubble’s current focus on, possibly even obsession with, transparency
and impact. You get the sense that many charity leaders believe that if we
could only display our accounts and give hard numbers on how many people we’re
helping, then the public and press would return to treating all charities as
angelic.
This is a limp hope. Few people have the time or inclination
to check the accounts and annual statements that charities painstakingly polished,
even fewer compare different possible donations in such detail. Even if they
do, they may not have the statistical grounding to make informed decisions, or
may leave with the wrong message, that all the charities they compared spend
too much on staffing, premises, IT and training. Certainly the sector should
not retreat on transparency, but nor should it slog on under the delusion that
once we reach a certain crystal-clear level, the public will fall in love.
Another difference charities may be more happy about. The
sector is far less regulated, and while the Charity Commission comes down hard
on some charities and may be seen as too bureaucratic, it pales in comparison
to pharmaceutical watchdogs. We have nothing that functions like the FDA/MHRA
testing and delaying new drugs for years. The Charity Commission does not
review every new project, grant or intervention that a charity plans, not even
very large experiments. Likewise most donors or funders would not be able to
block a charity functioning.
Try as we might we cannot create a ready-made academic
milieu for the voluntary sector, with the centuries of history, the
international network of journals, the expectation of challenge, refinement and
peer review. Multi-institutional multi-national collaborations do not spring up
overnight, but after years of relationship building, sharing techniques and
ethics, agreeing shared goals. But this is certainly a goal to have in mind:
through thought leadership, debate, seminars and working with the university
sector across disciplines, we must strive to introduce higher standards of
intellectual rigour and collective progress.
This article first appeared in Third Sector magazine, here.
Sir Stephen Bubb is
Director of Charity Futures. Jonathan Lindsell is the Research & Programme
Manager of Charity Futures.
