Tuesday 14 November 2017

Quality in children’s services – the central problem?

I missed a very important, if small scale, piece of research which was published about a year ago. So belatedly I have only just finished reading this very important, if underreported, piece of work. It points very clearly to what I think should be seen as the central problem of understanding issues of quality in children’s services in England.

The report in question [1], funded by the Nuffield Foundation and undertaken by researchers from the NSPCC, Loughborough University and the Child Outcomes Research Consortium, sets out the findings of a feasibility study into undertaking a larger project to try to understand how to define ‘good’ children’s social care services and how to assess if improvements occur.

It consists of two parts: a very useful and excellently reported literature review; and an analysis of the relationship between the Department for Education’s (DfE) outcome data for children and Ofsted ratings of children’s services.

The literature review section of the report concludes that:
  • There is a lack of consensus about what are good and what are bad outcomes for children’s social care services. 
  • There is no clarity about what indicators should be used to measure these outcomes.
  • There is only ‘mixed evidence’ about what characterises good children’s social care services and much of it is based on expert opinion rather than quantitative research.

Perhaps there are few surprises there, but having such a clear and systematic account provides an important baseline for future thinking.

Much more surprising is the analysis of the relationship between the Department for Education’s outcome data for children and Ofsted ratings of children’s services [2].

While one might reasonably expect to find that local authorities rated as ‘good’ or ‘outstanding’ by Ofsted also scored highly on the DfE’s outcome data [3], in fact the researchers found very little association between the two. Perplexingly, of the six local authorities which were ranked in the best 10%, according the DfE outcome data, only two were judged to be ‘good’ by Ofsted while two were found to be ‘inadequate’ and one ‘requiring improvement’ [4].

Looking at the data for all the local authorities, a regression analysis, showed that only one child outcome variable and one workforce variable had statistically significant relationships with Ofsted ratings and these associations were weak. Bizarrely the child outcome variable concerned was 'the percentage of looked after children who had a missing incident during the year'. The analysis showed a weak positive relationship indicating that the better the Ofsted rating the more missing incidents there were! The workforce indicator that had a weak statistically significant relationship with Ofsted ratings was the agency worker rate. Reassuringly this showed a negative relationship, with the lower the agency worker rate the better the Ofsted rating.

There were no statistically significant relationships between the other nine variables and the findings of the Ofsted inspections.

These are very disturbing results. 

The authors of the study put their findings before a seminar which was attended but what they describe as a variety of ‘experts’ from the DfE, Ofsted, the Association of Directors of Children’s Services, the Local Government Association, local authorities, the NSPCC and researchers from the various universities. The seminar, we are told, concluded with a ‘strong consensus’ that the DfE data and Ofsted ratings could not be relied upon to assess the quality of children’s social care services.

Just in case anybody thinks that we can just note these conclusions and move on, we need to be clear that they point to the systematic unreliability of at least one, and possibly both, of the main approaches used to measure the quality of children’s services in England. That is a major problem, a fundamental flaw.

Clearly both Ofsted and the DfE should take this very seriously. But there is no evidence that they are doing so. Ofsted’s Chief Inspector, Amanda Spielmann, was recently asked about Ofsted’s fitness to inspect children’s social care at a meeting of the House of Commons Education Select Committee, but she didn’t say anything about this research [5].

And as usual the DfE seems to apply a least-said-soonest-mended philosophy to communicating with the rest of us, so I could find nothing from them either.

To my mind the research points to a very hard rock in a very hard place, namely that the whole edifice of quality improvement in children’s social care is built on very shaky foundations.

I think there’s a mixed metaphor in that last sentence but frankly I don’t care!

Notes

[1] La Valle, I., Holmes, L., Gill, C., Brown, R., Hart, Di., Barnard, M. (2016).
Improving Children’s Social Care Services: Results of a feasibility study. London: CAMHS Press.

[2] Ofsted is the Office for Standards in Education, Children’s Services and Skills. It inspects and regulates services in England that care for children and young people, and services providing education and skills in England for learners of all ages. Ofsted’s inspectors carry out inspections of children’s services, including child protection, which rate individual local authorities as ‘outstanding’, ‘good’, ‘requires improvement’ or ‘inadequate’. https://www.gov.uk/government/organisations/ofsted/about

[3] As the government department responsible for children’s services in England, including child protection, the Department for Education’s (DfE) has amassed a ‘data set’ relating to the outcomes for children it believes to be important. These indicators measure Child Outcome Indicators (such as referrals within the past 12 months of a previous referral, repeat children protection plans, return home from care and emotional and behavioural health of looked after children) and Workforce Indicators (children in need per social worker, social worker turnover rate and agency worker rate). See La Valle et al (op cit.) Chapter 4 for more details.

[4] One inspection was incomplete.

[5] House of Commons, Education Committee, Tuesday 31st. October 2017