Saturday, 17 March 2012

Putting research in context

A good example of research being taken out of context seems to be in progress.

Professor Jane Ireland of the University of Central Lancashire carried out a study of a small number (126) of reports by “psychological expert witnesses” in Family Court proceedings. She found these to be of variable quality with two-thirds being rated by independent expert assessors as being of poor quality. Perhaps of greater concern, the research found that one in five of the so-called ‘expert’ psychologists was not properly qualified to act in this capacity.

The University of Central Lancashire’s website puts this research into appropriate context by quoting from a representative of the sponsors of the research, the Family Justice Council. This speaks of “expected methodological limitations” and a “small sample” for which “no claims are made as to its representativeness”. The sponsors describe the research as “useful” and as “an excellent signpost for future work, including reflection on how research methods can be refined in providing an initial view as to the quality of submitted expert psychological reports”.

These measured words are a far cry from the Daily Mail’s headline of an article that appears to be written by Professor Ireland herself.

“Family courts and how incompetent (but highly paid) so-called experts are failing children”

This article fails to explain that psychologists are but one of many groups of experts who provide evidence in the family courts. Nor does it explain that the research did not look at the quality of the evidence given by other groups such as paediatricians, radiologists, orthopaedic surgeons, psychiatrists, pathologists, forensic dentists … etc. etc.

Sadly the casual reader of the Daily Mail article is likely to conclude that most experts reporting in the family courts are incompetent, not the truth of the matter which is that a small proportion of one group of experts, psychologists, were found, in a small scale study, to be not properly qualified.

It is, of course, not surprising that Professor Ireland’s research as been seized upon by John Hemming MP, who campaigns vociferously on family court matters. Hemming has laid down an early day motion in the House of Commons that among other things says that Ireland’s research “… raises a massive question as to whether or not the judgments of the courts in the majority of care proceedings are reliable”.

But, of course, the research does nothing of the kind. It raises concerns about the quality of some evidence and it ought to be a spur to further research in this area. Possibly, it might entail an internal review of court practice. But a qualitative study of a small number of cases cannot purport to be representative and the results should not be generalised.

So let’s get things in perspective. There are some worrying things to which this research draws attention, but it is a small-scale study the results of which must be treated with caution. It does not raise “massive” questions nor does it cast doubts on the reliability of “the majority of care proceedings”.