Ofsted social care inspectors spend a lot
of time reading case files in order to make judgements about the quality of the
casework undertaken with children and young people.
Published sources are quite clear about
this. Ofsted’s inspection handbook and a document nattily titled “Framework and evaluation schedule: children inneed of help and protection and care leavers and Local Safeguarding ChildrenBoards” say much the same thing. Paragraph 61 of the handbook puts it like this:
“Inspectors will track the individual
experiences of at least 25 and usually no more than 30 children in need of help
and protection, children looked after and care leavers. In exceptional
circumstances the number of cases tracked may need to exceed 30 in order to
secure a representative judgement. The lead inspector should make a
proportionate decision about the number of additional cases to track. They will
take an in-depth look at the quality of the help, care and protection children
and young people have experienced and the implementation of children in need,
child protection, care, placement and pathway plans.”
Paragraph 65 of that document goes on to
say that the sample of children and young people whose cases are to be tracked
and sampled should be adjusted to ensure a balance of the following:
- age, gender, disability and ethnicity
- type of maltreatment/problem
- educational achievement and ability
- type of placement
- stages of “their journey”
- at least one child from a large sibling group
- practitioner and team
- children and young people supported by a third-party provider operating with social services functions delegated to it by the local authority
What all this fascinating detail fails to
show, however, is how the lead inspector ensures that the sample of cases is representative of the work undertaken by
the authority. How does s/he ensure that an undue proportion of either good or
bad work is not included in the sample? The answer to that question is clearly
crucial because if a biased sample is inspected, that will influence the
overall judgement of the inspection, effectively invalidating its findings.
The handbook clearly recognises that this
is an issue. Paragraph 96 says: “The lead inspector should make a proportionate
decision about the number of cases to sample in order to secure sufficient
evidence for a representative judgement.” But it doesn’t say how. And what is
meant by the phrase ‘proportionate decision’ is not explained. And Paragraph 61
says: “In exceptional circumstances the number of cases tracked may need to
exceed 30 in order to secure a representative judgement. The lead inspector
should make a proportionate decision about the number of additional cases to
track.” What these ‘exceptional circumstances’ are is not explained. Nor is
there any explanation of what securing a ‘representative judgement’ involves.
But clearly, we can all relax because that good old ‘proportionate decision’
will, no doubt, be made to resolve the issue!
That, you may be surprised to hear, is it. The
handbook says no more on the issue of how to ensure that the sample of cases
inspected is representative. There appears to be no further guidance to
inspectors. It’s over to them, I suppose.
It seems to me that this is very worrying.
In a post I made some time ago I looked at what I call the red bead problem. In that post I considered the problem of natural
variation and asked the question: “How can Ofsted be sure that the
variation between different local authorities, revealed in its inspections of
children’s services in England, is due to differences in performance rather
than just due to chance?”
Of course, I never got an answer from
Ofsted, probably because I expect nobody there reads my blog. I have eventually
got around to putting in yet another Freedom of Information Act request to
Ofsted, asking them how they ensure their samples are representative. So far,
they haven’t given me much of answer, but when they do I’ll keep you informed.
Watch this space.
