Ofsted social care inspectors spend a lot of time reading case files in order to make judgements about the quality of the casework undertaken with children and young people.
Published sources are quite clear about this. Ofsted’s inspection handbook and a document nattily titled “Framework and evaluation schedule: children inneed of help and protection and care leavers and Local Safeguarding ChildrenBoards” say much the same thing. Paragraph 61 of the handbook puts it like this:
“Inspectors will track the individual experiences of at least 25 and usually no more than 30 children in need of help and protection, children looked after and care leavers. In exceptional circumstances the number of cases tracked may need to exceed 30 in order to secure a representative judgement. The lead inspector should make a proportionate decision about the number of additional cases to track. They will take an in-depth look at the quality of the help, care and protection children and young people have experienced and the implementation of children in need, child protection, care, placement and pathway plans.”
Paragraph 65 of that document goes on to say that the sample of children and young people whose cases are to be tracked and sampled should be adjusted to ensure a balance of the following:
- age, gender, disability and ethnicity
- type of maltreatment/problem
- educational achievement and ability
- type of placement
- stages of “their journey”
- at least one child from a large sibling group
- practitioner and team
- children and young people supported by a third-party provider operating with social services functions delegated to it by the local authority
What all this fascinating detail fails to show, however, is how the lead inspector ensures that the sample of cases is representative of the work undertaken by the authority. How does s/he ensure that an undue proportion of either good or bad work is not included in the sample? The answer to that question is clearly crucial because if a biased sample is inspected, that will influence the overall judgement of the inspection, effectively invalidating its findings.
The handbook clearly recognises that this is an issue. Paragraph 96 says: “The lead inspector should make a proportionate decision about the number of cases to sample in order to secure sufficient evidence for a representative judgement.” But it doesn’t say how. And what is meant by the phrase ‘proportionate decision’ is not explained. And Paragraph 61 says: “In exceptional circumstances the number of cases tracked may need to exceed 30 in order to secure a representative judgement. The lead inspector should make a proportionate decision about the number of additional cases to track.” What these ‘exceptional circumstances’ are is not explained. Nor is there any explanation of what securing a ‘representative judgement’ involves. But clearly, we can all relax because that good old ‘proportionate decision’ will, no doubt, be made to resolve the issue!
That, you may be surprised to hear, is it. The handbook says no more on the issue of how to ensure that the sample of cases inspected is representative. There appears to be no further guidance to inspectors. It’s over to them, I suppose.
It seems to me that this is very worrying. In a post I made some time ago I looked at what I call the red bead problem. In that post I considered the problem of natural variation and asked the question: “How can Ofsted be sure that the variation between different local authorities, revealed in its inspections of children’s services in England, is due to differences in performance rather than just due to chance?”
Of course, I never got an answer from Ofsted, probably because I expect nobody there reads my blog. I have eventually got around to putting in yet another Freedom of Information Act request to Ofsted, asking them how they ensure their samples are representative. So far, they haven’t given me much of answer, but when they do I’ll keep you informed.
Watch this space.