In the last year Ofsted has got tough in its child protection inspections of local authorities, implementing a new inspection regime of unannounced visits.
As a result a substantial proportion of the authorities inspected are being rated ‘inadequate’. A statistical release for the period ending 31st December 2012 shows that overall 8 out of 23 of these inspections (35%) have resulted in a rating of ‘inadequate’ for overall effectiveness.
http://www.ofsted.gov.uk/resources/official-statistics-local-authority-childrens-services-inspections-and-outcomes
In 2013 two of the latest authorities to merit an ‘inadequate’ rating are Medway and Norfolk.
Michael Gove, the responsible government minister, has decided to talk-up these results. In November 2012 he is reported in Children and Young People Now as saying
“It is necessary to highlight how poorly some parts of local government are discharging their responsibilities.
But, of course, another way of looking at this is that the Government is presiding over a situation in which more than a third of local authorities are failing to provide an adequate child protection service. Surely, some might argue, such a high proportion of authorities failing these inspections constitutes a pressing reason for urgent government action, which to date is not forthcoming.
My own thoughts on these issues are mixed. While I am ready to accept that a substantial proportion of child protection services in England are seriously deficient, I have some doubts about Ofsted’s ability to produce objective evidence of this.
Last year I wrote to Ofsted under the Freedom of Information Act (2000) requesting information on the methodology used in their inspections of child protection arrangements. I was expecting quite a technical answer - with perhaps some weighty statistical manuals thrown in for good measure - and was surprised to be referred to just two publications [1] [2] which are available for download on their website.
So vague did I find these, with no real detail about how samples were selected or the mechanisms for preventing bias and eliminating subjectivity, that I wrote again to Ofsted asking them for the working documents, which I supposed were actually used by inspectors.
The answer I received shocked and surprised me. There were no other documents. Indeed, it seems that inspectors have considerable latitude in how they conduct their inspections and that there appears to be no statistical rigour applied to ensure consistency and objectivity.
Thinking about it, the absence of a tight methodology does explain a lot about Ofsted reports. It explains why I often find it difficult to see what the authority being inspected has done right (or wrong). It explains why judgements in reports often seem to me vague and imprecise. It explains why I often find the reports judgemental, rather than analytical.
I don’t think Ofsted has learnt much from Munro. Ofsted inspectors may have begun to talk about ‘decision-making’ more and about ‘timescales’ less, but they haven’t really got to grips with the idea that errors are not dealt with by catching the culprits and administering a slap on the wrist.
The lessons of Munro are that it is flaws in the design of services and organisations that pre-dispose towards error (‘error traps’ as Jim Reason calls them). Improvement consists in understanding where these are and how they can be eliminated – filling in the holes in Reason’s Swiss cheese slices [3].
There is no point in having a bunch of inspectors wandering around the country getting tough, if they are just flying by the seats of their pants and subjectively labelling one authority ‘adequate’ and another ‘inadequate’. In my view Ofsted needs to demonstrate that their inspections are doing more than this. And they need to move rapidly from a culture of spotting the ‘bad apples’, taking names and kicking arse, to a culture of safety, based on careful analysis and continuous improvement.
Endnotes
[1] “Framework for the inspection of local authority arrangements for the protection of children” http://www.ofsted.gov.uk/resources/framework-for-inspection-of-local-authority-arrangements-for-protection-of-children
[2] “Conducting inspections of local authority arrangements for the protection of children” http://www.ofsted.gov.uk/resources/conducting-inspections-of-local-authority-arrangements-for-protection-of-children
[3] See Reason, J. “Human error: models and management.” British Medical Journal 2000; 320:768-770 (18 March)
In 2013 two of the latest authorities to merit an ‘inadequate’ rating are Medway and Norfolk.
Michael Gove, the responsible government minister, has decided to talk-up these results. In November 2012 he is reported in Children and Young People Now as saying
“It is necessary to highlight how poorly some parts of local government are discharging their responsibilities.
But, of course, another way of looking at this is that the Government is presiding over a situation in which more than a third of local authorities are failing to provide an adequate child protection service. Surely, some might argue, such a high proportion of authorities failing these inspections constitutes a pressing reason for urgent government action, which to date is not forthcoming.
My own thoughts on these issues are mixed. While I am ready to accept that a substantial proportion of child protection services in England are seriously deficient, I have some doubts about Ofsted’s ability to produce objective evidence of this.
Last year I wrote to Ofsted under the Freedom of Information Act (2000) requesting information on the methodology used in their inspections of child protection arrangements. I was expecting quite a technical answer - with perhaps some weighty statistical manuals thrown in for good measure - and was surprised to be referred to just two publications [1] [2] which are available for download on their website.
So vague did I find these, with no real detail about how samples were selected or the mechanisms for preventing bias and eliminating subjectivity, that I wrote again to Ofsted asking them for the working documents, which I supposed were actually used by inspectors.
The answer I received shocked and surprised me. There were no other documents. Indeed, it seems that inspectors have considerable latitude in how they conduct their inspections and that there appears to be no statistical rigour applied to ensure consistency and objectivity.
Thinking about it, the absence of a tight methodology does explain a lot about Ofsted reports. It explains why I often find it difficult to see what the authority being inspected has done right (or wrong). It explains why judgements in reports often seem to me vague and imprecise. It explains why I often find the reports judgemental, rather than analytical.
I don’t think Ofsted has learnt much from Munro. Ofsted inspectors may have begun to talk about ‘decision-making’ more and about ‘timescales’ less, but they haven’t really got to grips with the idea that errors are not dealt with by catching the culprits and administering a slap on the wrist.
The lessons of Munro are that it is flaws in the design of services and organisations that pre-dispose towards error (‘error traps’ as Jim Reason calls them). Improvement consists in understanding where these are and how they can be eliminated – filling in the holes in Reason’s Swiss cheese slices [3].
There is no point in having a bunch of inspectors wandering around the country getting tough, if they are just flying by the seats of their pants and subjectively labelling one authority ‘adequate’ and another ‘inadequate’. In my view Ofsted needs to demonstrate that their inspections are doing more than this. And they need to move rapidly from a culture of spotting the ‘bad apples’, taking names and kicking arse, to a culture of safety, based on careful analysis and continuous improvement.
Endnotes
[1] “Framework for the inspection of local authority arrangements for the protection of children” http://www.ofsted.gov.uk/resources/framework-for-inspection-of-local-authority-arrangements-for-protection-of-children
[2] “Conducting inspections of local authority arrangements for the protection of children” http://www.ofsted.gov.uk/resources/conducting-inspections-of-local-authority-arrangements-for-protection-of-children
[3] See Reason, J. “Human error: models and management.” British Medical Journal 2000; 320:768-770 (18 March)
