A Response to the blog by Puebla and Heber of PLOS ONE

PLOS ONE has issued an expression of concern regarding a cost-effectiveness analysis on results from the PACE trial. The concern is because the authors will not provide data requested by Professor Coyne. The PLOS ONE regulations in force at the time stated:

‘Publication is conditional upon the agreement of the authors to make freely available any materials and information described in their publication that may be reasonably requested by others for the purpose of academic, non-commercial research.’

Queen Margaret University London (QMUL) is the responsible authority for the PACE trial, but for the purposes of this particular paper the role is played jointly by Kings College London (KCL) and QMUL. KCL and QMUL continue to refuse to release the data.

Iratxe Puebla, Managing Editor for PLOS ONE, and Joerg Heger, Editor-in-Chief of PLOS ONE, have written a blog giving their view of the arguments involved and an insight into some of their thinking in issuing the expression of concern.

One line gives cause for concern and I have written a response. I did post this response as a comment underneath their blog, but after more than 24 hours the comment still remains ‘awaiting moderation’. I have therefore decided to publish it here.

It is worth reading also the dismissive, arrogant, ignorant letter to Professor Coyne from KCL refusing his request for the data.


(Note: the comment has now been approved on the PLOS ONE site.)


My response:

“Interestingly, the ruling of the FOI Tribunal also indicated that the vote did not reflect a consensus among all committee members.”
This line is misleading and reveals either ignorance or misunderstanding of the decision in Matthees.

The Information Tribunal (IT) is not a committee. It is part of the courts system of England and Wales.

First, the IT’s decisions may be appealed to a higher court. As QMUL chose not to exercise this right but to opt instead to accept the decision, then clearly it considered there were no grounds for appeal. The decision stands in its entirety and applies without condition or caveat.

Second, court decisions are not applied differently according to how those decisions are reached: they are full and final. Majority verdicts have no less standing. We are all familiar with the work of the UK & US Supreme Courts. Roe v Wade is not mitigated because it was a majority decision. May could not fudge the need for parliamentary approval of Brexit because the UKSC was not unanimous.

Third and above all, it is misleading to suggest there was a lack of consensus in the Tribunal.
The court had two decisions to make:
First, could and should trial data be released and if so what test should apply to determine whether particular data should be made public? Second, when that test is applied to this particular set of data, do they meet that test?

The unanimous decision on the first question was very clear: there is no legal or ethical consideration which prevents release; release is permitted by the consent forms; there is a strong public interest in the release; making data available advances legitimate scientific debate; and the data should be released.

The test set by this unanimous decision was simple: whether data can be anonymized. Furthermore, again unanimously, the Tribunal stated that the test for anonymization is not absolute. It is whether the risk of identification is reasonably likely, not whether it is remote, and whether patients can be identified without prior knowledge, specialist knowledge or equipment, or resort to criminality.

It was on applying this test to the data requested, on whether they could be properly anonymized, that the IT reached a majority decision.

On the principles, on how these decisions should be made, on the test which should be applied and on the nature of that test, the court was unanimous.

It should also be noted that to share data which have not been anonymized would be in breach of the Data Protection Act. QMUL has shared these data with other researchers. QMUL should either report itself to the Information Commissioner’s Office or accept that the data can be anonymized. In which case, the unanimous decision of the IT is very clear: the data should be shared.

PLOS ONE should apply the IT decision and its own regulations and demand the data be shared or the paper retracted.

A Response to Esther Crawley

This is in response to the comment piece in New Scientist by Esther Crawley.


Myalgic Encephalomyelitis (ME) has been studied for decades from Ramsay through Behan to Hornig and Newton. As a result, many would say we know rather a lot about the illness. We know that patients show neurological, immunological and endocrinological changes. We know that ME is not depression; that patients do not respond more to placebo; that patients do not fear exercise; that ME is not caused by physiological deconditioning. We also know that patients do not harass researchers and that the primary symptom is post-exertional malaise.

It is not true that patients believe clinicians secretly think the illness is psychological. There is no secret: the major proponents of the Cognitive Behavioural Therapy-Graded Exercise Therapy (CBT-GET) model say ME is a self-perpetuating cycle of exercise avoidance. They have all stated quite clearly that ME has no ongoing underlying biological cause: it is neurasthenia, ‘simply a belief’, a functional somatic syndrome (Simon Wessely); it is perpetuated by beliefs (Peter White); ME is a pseudo-disease, a somatoform disorder, perpetuated by misinterpretation of bodily sensations, abnormalities of mood and unhelpful coping behaviour (Michael Sharpe). If others, such as Dr Crawley, disagree and do not mean to imply that ME itself has a psychological element, then perhaps they could say so unambiguously.

While it is of course true that anything which brings about changes is ‘biological’, implicit in CBT is the notion that responsibility for recovery lies with the patient. If only patients think differently, then they will no longer be ill. No one disputes psychotherapy can help with a broad spectrum of illnesses, but there is no evidence it can reverse organic damage (injury, infection, inflammation). There is no evidence CBT can address the changes in ME patients found by Lipkin, Montoya and Naviaux.

It is true that ME is likely to prove to be more than one illness, or an illness with more than one sub-type. It is also true though that many people diagnosed with ME, do not in fact have it. This difficulty in diagnosis due to the absence of a biomarker is one which causes problems for clinical trials. Many patients think the criteria used by Dr Crawley are too broad and are likely to include patients who have a generic ‘chronic fatigue’. It is a view shared by the US Institute Of Medicine and the US Agency for Healthcare Research and Quality, which has recently stopped recommending CBT for ME because any claims for its efficacy come from trials which used the discredited Oxford criteria.

The other challenge for trials of interventions for ME is to distinguish between placebo, improved coping with the effects of the illness and a genuine treatment of the underlying illness. Since the severely ill do not respond at all to CBT and the small, subjective, self-reported benefit in the moderately ill is only temporary, many patients think that claims for effectiveness of CBT are unsafe. They are not convinced FITNET-NHS contains sufficient safeguards against this confound.

Whichever measurements of recovery and improvement are used, whether the ones initially chosen by the investigators or the ones to which they switched part-way through the trial when some data had already been collected, we do know from PACE that the vast majority of patients do not benefit at all from CBT-GET. Since NHS clinics are based on this approach, many patients do not bother attending them. Any trial, then, gathering subjects from these clinics would not only exclude the most severely ill, who are unable to travel, but also the large numbers of patients who gain no advantage from the interventions and so do not waste time and effort in going to the clinics. On the other hand, since these clinics use broad criteria to select for ME, patients fear that people with illnesses other than ME would be included.

Patients agree that it is important we all work together. We would ask, though, that any research uses strict criteria to exclude chronic fatigue; takes into account the physiological changes found in people with ME; is based on plausible theory; is not based implicitly or explicitly on the notion the illness is one of false beliefs; includes meaningful patient involvement, from the broader patient community not just the established charities; benefits all patients, including the most severely affected; has, where applicable, proper controls against confounds; and unconditionally shares all anonymized data with anyone who wants to see it.


Thanks to (& #FF) Samei Huda for advice, though his help should not be seen as any kind of endorsement or agreement.

Sense About Science, the PACE trial and ME.

(The email exchange discussed in this blog can be viewed here: https://justpaste.it/xnq8 )

Sense About Science (SaS) exist to challenge misrepresentation of science and evidence. They advocate openness and honesty about research findings. They encourage people to #askforevidence. They agree that: we need all available information to make informed decisions about health care; hiding half the data is how magicians do coin tricks and shell games; with incomplete data we can only get an incomplete picture; outcome switching is like choosing lottery numbers after watching the draw. They ask people to contact them when there is something wrong so they can make a fuss.

When patients were trying to obtain the PACE trial data from Queen Mary University of London, many of us asked SaS if they would help us. They refused. They said that the trial results were available and that the investigators had complied with CONSORT.

In March, Rebecca Goldin posted a scathing criticism of PACE on the website of stats.org concluding that ‘the flaws in this design were enough to doom its results from the start’. In an accompanying editorial Trevor Butterworth of Sense About Science USA, equally as critical, said that ‘the way PACE was designed and redesigned means it cannot provide reliable answers to the questions it asked’.

Sense About Science USA is described as the sister organization of SaS. It runs the stats.org website in collaboration with the American Statistical Society .

After such criticism of PACE by Sense About Science USA and stats.org, would SaS, the UK organization, now support patients? They gave no signal they would, so in June I emailed SaS. I got an automatic response acknowledging receipt of my email, but no reply. I waited a few weeks and tried again. The same thing happened. In July, I wrote a letter to Tracey Brown. She didn’t even have the good manners to reply. In September I emailed Professor Paul Hardaker, chair of the trustees, asking if he could help me get an answer to my questions. He replied almost immediately, apologized and passed on my email. Soon after, Julia Wilson sent me an email.

I asked five questions of SaS:

Do they accept Goldin’s analysis of PACE and Butterworth’s criticism as valid?

Why, despite our requests, had they not made a fuss about something said by stats.org to be wrong? Would they?

Could they say where the data were available, the ‘results’ of the PACE trial?

Did they support the attempt by the PACE investigators to extend the Data Protection Act to prevent sharing of trial data?

Would they allow us a right of reply to Michael Sharpe’s interpretation of his own study which they had been carrying on their site since last October?

By the time I received a reply the Tribunal decision had been made, ordering QMUL to release the data. My third question was redundant.

SaS conceded Sharpe’s piece contravened their editorial policy and added a rider to that effect on their website. They claimed that they had never supported the extension of the DPA, but had not had enough resources to help in the case. Not enough it seems to post a tweet or send a single email.

They did not say whether they accepted the analysis and criticism on the stats.org website as valid.

There then began an exchange in which Wilson singularly failed to answer simple questions and continued to use language like a politician trying to avoid an issue. They did, though, remove the part of Sharpe’s article in which he made claims for his own study.

Eventually Wilson stopped responding to my emails, so I copied in Hardaker again. This time she did reply and she did finally state that SaS accepted Goldin’s analysis as valid. According to SaS the PACE trial is flawed. Wilson then ended the exchange.

They still refuse to help in any way. They have not welcomed the Tribunal decision. Even though they agree PACE is flawed, they are not prepared to do anything about it. They do not say whether they accept Butterworth’s criticism as valid.

A number of questions remain for SaS:

Why did they ignore my emails and only answer when I contacted the chair of the trustees?

Why did they allow Sharpe to promote his own study, contrary to their own editorial policy?

Why did they not push for the release of PACE trial data?

Why did they not support patients in their case against QMUL when QMUL were attempting to extend the DPA, which would have had a stifling effect on trial transparency generally?

Why have they never welcomed the Tribunal decision?

Why did they take so long and why were they so reluctant to say they accept Goldin’s analysis as valid?

Why will they not say they accept Butterworth’s criticism as valid?

Why do they still refuse to make a fuss about PACE?

Why is it that for SaS different rules apply when it comes to ME?

Using public money to keep publicly funded data from the public

Update. Some have questioned whether QMUL had to pay VAT, or whether the VAT could be reclaimed by QMUL. I have done a further FOIA request to clarify and received a response on 14/09/16:

‘VAT at 20% was paid on these amounts.’


After publication of the PACE trial comparing different interventions (Cognitive Behavioural Therapy [CBT], Graded Exercise Therapy [GET], Adaptive Pacing Therapy and Specialist Medical Care) for ME/CFS, patients questioned the claims for the effectiveness of CBT and GE. These criticisms have been reported by David Tuller and James Coyne (a series on his blog here), and supported in her own look at the trial by Rebecca Goldin for the website jointly run by the American Statistical Association and Sense About Science America.

A number of Freedom of Information requests were made for the data in order to test the conclusions drawn by the Principal Investigators. Many of the patients’ requests were rejected, deemed vexatious, by the responsible research centre, Queen Mary University of London (QMUL).  In one instance, however, Alem Matthees successfully complained to the Information Commissioner (IC), and QMUL were ordered to release the data Matthees had requested.

QMUL appealed the IC’s decision and a hearing of the First-Tier Tribunal (Information Rights) was held in April this year.

Valerie Eliot Smith, a qualified barrister, has done a series of blogs on the hearing and I am grateful to her for information used here. On her website a number of downloads are available, including one which lists the attendees (bottom of page here).

Those at the Tribunal for QMUL include: a QC, Timothy Pitt-Payne; a solicitor, Edward Hadcock; two assistant solicitors, Alison Williams Mills and Gary Attle; four witnesses, Peter White (QMUL), Steve Thornton (QMUL), Trudie Chalder (KCL) and Ross Anderson (Cambridge); and two observers, Jane Pallant (Deputy Academic Registrar) and Paul Smallcombe (FOI Officer).

Hiring a QC for three days does not come cheap. I made an FOI request to QMUL to discover exactly how much the hearing had cost and have now received a reply.

It is claimed that the attendance of all those witnesses and any preparation involved for the Tribunal cost the University nothing. Presumably Anderson covered his own travel costs and the attendance at and preparation for the hearing by White and the others were considered part of normal work duties.

QMUL has said how much they paid in legal fees:

Mills & Reeve LLP: £149,482.30 ex VAT
Timothy Pitt-Payne QC: £48,320.00 ex VAT
Disbursements/expenses (Mills & Reeve LLP): £6,985.43 ex VAT

VAT is charged at 20% on legal fees. I make the total amount of public money QMUL has so far spent to keep data secret:



Screenshot of email from QMUL:


Why the dispute about PACE trial data matters to everyone.


Queen Mary University of London are appealing an Information Commissioner decision that the data for the PACE trial  should be released.

The Principal Investigators for that trial are also trying to claim that the PLOS One data-sharing regulations should not apply.

Since just about everything to do with Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS) is disputed, it’s no surprise that there has been controversy surrounding the PACE trial from the start. But now this argument threatens to become much wider and drag in a large part of academic research, any study, in fact, in any field, that involves human participants.

The publicly funded PACE trial, published in 2011 in The Lancet, compared different interventions (Cognitive Behavioural Therapy, Graded Exercise Therapy, Adaptive Pacing Therapy and Specialist Medical Care) for ME/CFS. Since its inception the study has been criticized by patients, and after its publication they made a series of Freedom of Information requests for the data. The responsible research centre, Queen Mary University of London, has received 35 requests for 160 pieces of information. Many have been rejected, deemed vexatious, on the grounds that patients were waging a campaign to discredit the trial.

This dispute, largely until that point between patients and the investigators, escalated at the end of last year after a series of blog posts by David Tuller and the intervention of Professor James Coyne. Coyne, who has a long history of exposing and debunking pseudoscience, made his own request for the data from the publishers of a PACE follow-up study on the cost effectiveness of the interventions. This study was not in the Lancet but carried by PLOS One, which has a stated policy of data sharing. Coyne applied under the PLOS One regulations in force at the time of publication.

At first, Coyne was also told his request was being rejected as vexatious. This was clearly absurd: Coyne is well respected with an excellent reputation, indeed could be argued as having a bigger career reach than any of the trial investigators. It was then suggested that trial data could not be properly anonymized. This reason was in turn dropped: a sibling trial, FINE, had shared data, and the FOI commissioner recently ruled on another PACE request by patients that the data could be anonymized. Peter White, the most senior of the trial principal investigators (PIs), conceded in a recent letter to the Wall Street Journal that the data can be anonymized.

White now claims, as stated in the letter, that they must safeguard the data to protect the patients’ interests; they have a duty of care such that any request for data must be in the context of a defined research proposal deemed acceptable by institutional committee. It is this claim that, were it to succeed, would have a stifling effect on academic research in many fields. If any such claim were recognized, it would not just apply to this trial but be a precedent applied across disciplines to any studies involving human participants.

It must not be allowed to succeed.

It is also false.

First, institutional committees, while no doubt full of people with good intentions, are liable to be over-respectful to their colleagues. They are more likely to support the PIs’ views of what is deemed proper. They would be a potential brake on challenging false reports. Trial investigators would effectively be granted a veto as to who could see their data.

Second, ‘defined research proposals’ need funding and often an infrastructure to support them. Such a requirement would create a closed shop in academic research: the ‘citizen scientist’, the curious student and the lone professor would all be excluded. Often advances come not from planned studies but from discoveries revealed by ‘playing around’ with data. That avenue would be shut off.

Third, this interest, which the PIs claim patients have in data from trials in which they have participated, does not exist. Participants do have certain rights, including a right to know of investigators’ conflicts of interest, something which was not revealed to patients in the PACE trial. They have the right to be treated humanely throughout the trial and to have their health and well-being protected. They have the right to know that their personal details will be secure and any shared data will be properly anonymized, such that no one, for example, could reverse engineer the data and reveal their identity. But once such rights have been safeguarded, then the data are just numbers. There is nothing further to be protected. Investigators cannot invent some duty of care which would give them the right to lock away data in perpetuity, sacred runes that only selected High Priests could see.

Furthermore, if a public interest does exist, then it is in having the data made available as widely as possible. Participants’ time and effort should be properly rewarded by making the greatest use of the data. They should be confident that any analysis of that data, including by the trial investigators, is open to the widest and deepest scrutiny; that, in fact, investigators are not allowed to protect themselves by putting up artificial barriers to inspection of the data. This is an interest shared by the wider society: open science benefits us all. Such an interest is even greater when the data have been obtained by public funding, as with the PACE trial.

Data do not need to be protected. Science, all academic study, is eventually self-correcting. We should have confidence in the robustness of the scientific process, an essential element of which is the greatest possible openness. Paternalistic protection of rows of numbers serves no one other than those with something to hide.

If the PIs are allowed to create this duty of care based on participants’ interest, then every trial investigator will be free to claim it. Drug companies and homeopaths will be able to hide behind this wall, but so will anyone who has conducted a trial using human subjects in medicine, science, psychology, economics, law, philosophy… Every field.

The PACE trial has had a massive impact. It has shaped public policy and perception of the illness in the UK. It has been referenced by health bodies around the world. The findings and the model it purports to evidence have been supported by Professor Sir Simon Wessely, the Lancet and its editor, Richard Horton, NICE, NHS Choices, the Science Media Centre, Sense About Science (UK) and most of the British media, including, for example, Tom Feilden of the BBC.

Patients who challenged the study and rejected the illness model were portrayed as science deniers who were unwilling or unable to accept the truth of their illness, desperate to avoid the stigma of mental illness, erroneously clinging to a false mind/body dualism. They were accused of waging personal vendettas against the PIs and Wessely in particular.

Then about 18 months ago things started to change: separate reports, here and here, to the US NIH and IOM rejected the psychogenic model and pretty much supported everything patients have been saying for decades. Prominent scientists, Lipkin and Hornig from Columbia, Montoya from Stanford and Komaroff from Harvard Universities have been investigating the illness and finding more and more evidence of physiological changes in patients. The NIH has recently announced a multi-million dollar project to study ME as a biological illness. After a small but successful trial of Rituximab, an editorial in New Scientist talked of the scientific story’s being brought full circle, back to the ‘post-viral’ approach of 25 years ago. Alongside Tuller’s blogs and Coyne’s interventions, 40 doctors and scientists wrote an open letter to the Lancet asking for the PACE trial data to be released for analysis. Amongst others, neuroscientist Keith Laws, statistician Andrew Gelman and biologist and campaigner against scientific fraud Leonid Schneider have all posed questions about PACE and its follow-up studies. Professor of mathematical sciences Rebecca Goldin took a detailed look at PACE for  the website jointly run by the American Statistical Association and Sense About Science America. Her conclusion was that ‘flaws in this design were enough to doom its results from the start’.

It is estimated that 250,000 patients suffer from ME in the UK. For their sake, the data need to be released and the questions over the trial answered. For everyone’s sake, the PACE PIs cannot be allowed to introduce a restriction on access to data that would act as a massive constraint on a great deal of academic research.