The Borna Disease Virus Tragedy

The Robert Koch Institute in Berlin (remember Koch’s Postulates?) earlier this year cancelled its research into Borna Disease Virus (BDV). That is really regrettable, because BDV, a neurotropic virus, may cause depression and bipolar disorder. BDV may be transmitted by blood transfusions; research on this is being carried out in Australia. There may even be an effective cure; an antiviral drug, amantadine sulfate, approved for use against the common flu.

Given the prevalence of depression and bipolar disorder, you should have thought that a decent randomized trial would have been carried out. A small trial would not cost more that a couple of hundred thousand dollars. But amantadine is a 30 year old drug, no longer covered by patents. There is practically no incentive for pharmaceutical firms to finance such a study. So far there has been none. Given the potential huge benefits and the low cost, that is a tragedy.

In the meantime, German and Austrian doctors are using amantadine and reportedly getting good results. However, we still need the solid evidence we would get from a well-designed randomized controlled trial.

For background, read this article in Discovery, a more recent paper here. See also

Where are the Systematic Reviews we need?

New Scientist writes (subscription necessary to read article),

If you want to know how to preserve biodiversity, do not rely on articles in conservation journals, a new study warns.

IF YOU want to know how to preserve biodiversity, don’t rely on articles in conservation journals. So says a study which argues that conservationists should follow the medical profession’s lead, and ensure that their decisions are objectively based.

“We’re about 30 years behind the medical revolution,” says Philip Roberts of the Centre for Evidence-Based Conservation at the University of Birmingham in the UK. As a standard to aspire to, Roberts and his team took the systematic reviews that are the bedrock of evidence-based medicine. These reviews start with a carefully framed question, and typically list the search terms used to find the studies to be analysed. Strict criteria are then applied to exclude poor-quality research, and finally rigorous statistical tests on the pooled results are used as the basis of an objective guide for doctors to what treatments work best.

The study referred to is Are review articles a reliable source of evidence to support conservation and environmental management? A comparison with medicine.

Abstract,

Useful and interesting blogs

Take a look at these two blogs on evidence-based practice, Bob Sutton’s Work Matters and Tracy Allison Altman’s Evidence Soup.

Bob Sutton is a professor of management science and engineering at Stanford. He writes about Evidence Soup, “Reading Altman’s blog is like taking an ongoing course in how to make evidence-based decisions and how to take – and evaluate – evidence based actions.”

I have previously mentioned Bob Sutton as the co-author of Hard Facts, Dangerous Half-Truths And Total Nonsense: Profiting From Evidence-Based Management.

Read e.g. his Management Advice: Which 90% is Crap? (this is a reference to John Wanamaker’s statement that 50% of the money he spent on advertising was wasted, but he didn’t know which 50%).

Evidence-based medicine

I have previously mentioned evidence-based medicine. Here is a cover story on the subject from Business Week.

Data quality problem in systematic review

J.S. Brooks et al. DEVELOPMENT AS A CONSERVATION TOOL: EVALUATING ECOLOGICAL, ECONOMIC, ATTITUDINAL, AND BEHAVIORAL OUTCOMES: REVIEW REPORT is an excellent attempt at a systematic review. Unfortunately, there is a serious problem with lack of quality data.

Main Results
The results of this review are that (1) very few studies provide adequate quantitative measures of success across multiple outcomes to provide a strong test of the hypotheses, and (2) that two separate statistical approaches to the data indicate market selling opportunities are associated with attitudinal outcomes, and community involvement in decision making and implementation is associated with behavioral success.

Conclusions
As regards the first objective, it is clear that without far better monitoring schemes in place it is still impossible to provide a systematic evaluation of how different strategies are best suited to different conservation challenges. First, there is a paucity of high quality data. Second, few studies provide quantitative evaluations of success. Third, few studies evaluate across the full range of relevant outcomes – behavioral, attitudinal, economic and ecological…

This and other papers can be found at the Centre for Evidence-Based Conservation at the University of Birmingham, U.K.

Gloom & Doom, Inc.

If a doom-monger came up with a set of recommendations that would imply less money and power to the class of people to whom he or she belongs, I would at least listen with interest.

We shouldn’t listen to the latest “obey us, or face doom” message.

A recent commentary in Nature (subscription necessary) calls for an international body of biodiversity experts.

…it should have a formal link to, and be funded by, governments. This feature, which distinguishes it from previous biodiversity initiatives, would ensure that negotiations within international biodiversity conventions are based on validated scientific information and lead to action at national and global levels.

So, because governments would (use taxpayers’ money to) pay for this, that would ensure action “at national and global levels”? And this “action” would be effective? The authors of the proposal owe a payment for ecosystem services to the owners of the ecosystem that produced whatever they smoked.

The consultation process, supervised by an international steering committee, will last 18 months and proceed in two phases. During the first phase, a number of studies will define the need for, and goals of, an international panel on biodiversity. These studies will examine the global decision-making landscape concerned with biodiversity, analyse successes and failures of biodiversity conservation efforts at different scales, and assess existing international mechanisms that deliver scientific expertise. In a second phase, this information will be used to articulate a set of recommendations for an international panel, which will be presented at a set of regional meetings to seek input from all sectors of society and all regions of the world.

This is not about action, or doing science, this is about creating and getting money for a talking shop.

We urgently need a scientific body of knowledge on conservation. What works, what doesn’t work, what are the cost and the benefits? We can’t do systematic reviews before the field trials have been done. Diverting scientists away from science and getting funds for scientists to pontificate on the basis of weak science is a waste both of scientists and of money.

Evidence-based interventions

Examples of systematic reviews, from the Cochrane Collaboration’s recent newsletter,

• Don’t bother with intravenous rehydration for diarrhoea – oral rehydration works just as well

In wealthy countries it is fashionable to prefer intravenous therapy (IVT) over oral rehydration therapy (ORT). A Cochrane Review published, however, shows that ORT is just as effective as IVT.

• Treating water at home is effective in preventing diarrhoea, a major cause of death in young children in developing countries

Supplying clean water to a community helps reduce gastrointestinal diseases, but interventions that kill disease-causing waterborne micro-organisms (or microbes) once it has reached the home can be even more effective. These are the conclusions of a systematic review that considered the outcome of 38 field trials involving more than 53,000 participants.

Before you can carry out systematic reviews, you must have well-designed studies to review. Then you can generalize from those studies. “38 field trials involving more than 53,000 participants”. Sigh. How many useful, well designed field trials do we have in conservation?

Reference-class forecasting

One of the cures for optimism bias, or what Flyvbjerg (see previous post) less diplomatically calls “lying”, is reference-class forecasting.

The idea is simple, see this brief article by Lovallo & Kahneman, and the paper by Flyvbjerg, Procedures for Dealing with Optimism Bias in Transport Planning: Guidance Document,

(1) Identification of a relevant reference class of past projects. The class must be broad enough to be statistically meaningful but narrow enough to be truly comparable with the specific project.

(2) Establishing a probability distribution for the selected reference class. This requires access to credible, empirical data for a sufficient number of projects within the reference class to make statistically meaningful conclusions.

(3) Compare the specific project with the reference class distribution, in order to establish the most likely outcome for the specific project.

Underestimated costs + Overestimated benefits = Project approval

Bent Flyvbjerg has built up a large database with information about costs and benefits of large infrastructure projects. He found, for example, that on rail construction projects, cost overruns are on the average 44.7%, and actual traffic is 51.4% lower than estimated traffic. What explains this? Flyvbjerg writes,

Underestimated costs + Overestimated benefits = Project approval

Using this formula, and thus “showing the project at its best” as one interviewee said above, results in an inverted Darwinism, i.e., the “survival of the unfittest.” It is not the best projects that get implemented, but the projects that look best on paper. And the projects that look best on paper are the projects with the largest cost underestimates and benefit overestimates, other things being equal. But these are the worst, or “unfittest,” projects in the sense that they are the very projects that will encounter most problems during construction and operations in terms of the largest cost overruns, benefit shortfalls, and risks of non-viability. They have been designed like that…

Strategic misrepresentation can be traced to political and organizational pressures, for instance competition for scarce funds or jockeying for position, and it is rational in this sense. If we now define a lie in the conventional fashion as making a statement intended to deceive others …, we see that deliberate misrepresentation of costs and benefits is lying, and we arrive at one of the most basic explanations of lying that exists: Lying pays off, or at least political and economic agents believe it does.

Where, oh where, are the studies we need?

Can we work to reduce poverty and conserve biodiversity at the same time? In a recent paper (Poverty, Development, And Biodiversity Conservation: Shooting in the Dark?) Arun Agrawal and Kent Redford write

the mass of scholarly work on the subject does not permit systematic and context-sensitive generalizations about the conditions under which it may be possible to achieve poverty alleviation and biodiversity conservation simultaneously. The vast sums channeled toward joint achievement of poverty alleviation and biodiversity conservation are all the more remarkable in light of the basic lack of evidence on the extent to which these goals can jointly be reached…

Better research design, based on careful specification of the relevant hypotheses, will likely require panel data from a suite of sites and households to allow systematic comparison across cases and regions…

…before and after studies are likely to prove invaluable in gaining a deeper understanding of the links between different measures of poverty and biodiversity.

For a relevant article, read also Jon Christensen’s Win-Win Illusions. I recommended it.

Lessons Learned

Today this blog is one month old. Let’s look back at some lessons learned and at some known unknowns.

There is a urgent need for evidence-based conservation. We should learn from the medical profession, and use randomized, controlled trials to validate interventions. Why not get experts in clinical trials involved in the design of conservation studies? Once we have a number of solid studies, we can carry out systematic reviews.

You can’t understand the pressures on protected areas, and the pro or contra-conservation activities outside protected areas, if you don’t calculate the cost and benefits to the people who live in the area. Such a calculation necessarily involves calculating land rents and opportunity costs.

In poor countries, benefits may largely accrue to foreign hotel and tour operators, or to the central government, leaving local people with scant incentive to conserve nature. What is the mechanism that maintains this system? Public choice theory is an analytic framework that could be useful in understanding the choices of governments and NGOs.

Protected areas work, but the Galapagos Effect is real. A low human population density is good for conservation. An attractive protected area might attract so many visitors, as well as people earning their living from visitors, that the protected area suffers (there is a second Galapagos Effect in biology; it refers to rapid evolutionary change in small isolated populations under strong selection pressure).

Direct payments for conservation is a very promising innovation. You get what you pay for, and if you don’t pay for conservation, you shouldn’t be surprised if you don’t get it. Projects with multiple objectives and indirect payments are wasteful, but can be understood in terms of the incentive structures of NGO and government bureaucracies. Direct payments became common in disaster relief during the 1990s. Why not in conservation?

Conservationists and local people do not have common preferences, but cooperative bargaining can provide acceptable solutions to both parties.

ITQs, individual transferable quotas, have worked in fisheries, water supply, and pollution control. Can their use be extended? Are there cases where we can show that they can’t work?

Land trusts have grown very fast in the U.S.A. over the past half century. Can they be replicated outside the U.S.?

Evidence-based conservation?

In medicine, the Cochrane Collaboration has pioneered evidence-based medicine through the use of randomized trials and systematic reviews. “Evidence based-medicine?” Does that mean that the practice of medicine is not totally based on evidence? Well, yes. It is a horrifying thought that the medical treatment you receive may not, in fact, be based on real evidence. But at least the medical profession is working on it.

The evidence-based approach has spread to social interventions. The Campbell Collaboration (C2) “is a non-profit organization that aims to help people make well-informed decisions about the effects of interventions in the social, behavioral and educational arenas. C2’s objectives are to prepare, maintain and disseminate systematic reviews of studies of interventions.”

In the field of management, there is a new book by two very well know business school professors, Jeffrey Pfeffer and Robert Sutton, Hard Facts, Dangerous Half-Truths and Total Nonsense: Profiting from Evidence-Based Management.

What is going on in conservation? In a recent paper (hat tip to Marcelino Fuentes) (Money for Nothing? A Call for Empirical Evaluation of Biodiversity Conservation Investments), Ferraro and Pattanayak lament the fact that the field of ecosystem protection and biodiversity conservation lack behind most other policy fields.

They write, “The field of conservation policy must adopt state-of-the-art program evaluation methods to determine what works and when. How many elephants would be poached if there had been no law banning ivory trade?”

Good question. Half-truths and slogans have done a lot of damage in Africa, for example in the militaristic approach to “anti-poaching “. Beware of any organization that wants to collect money for trucks, radios and guns to wage war on the local population in order to protect wildlife. Demand that they show how their projects will make the local “poacher” better off by conserving rather than harvesting wildlife.

There are the beginnings of the evidence-based approach in conservation. For example, the Centre for Evidence-Based Conservation at the University of Birmingham, U.K. “was established in 2003 with the goal of supporting decision making in conservation and environmental management through the production and dissemination of systematic reviews on the effectiveness of management and policy interventions.”