As suppliers of Digital Health systems gradually comply with the SCCI 0129 safety standard, healthcare organisations are increasingly finding themselves on the receiving end of Hazards Logs and Safety Cases. But what should they do with them? SCCI 0160 clearly requires healthcare organisations to undertake their own risk assessment but the Standard says remarkably little on how the supplier’s materials should be used.
At Safehand, the first thing we recommend is that the healthcare organisation should think about whether the supplier’s Safety Case stacks up. Is it believable? Do you have confidence in its findings? It is objective or contrived?
Here’s our CHIME framework, a useful acronym for the top 5 things to look for in a good Safety Case:
Credibility – Is what you are seeing believable? Is the Safety Case a hasty retro-fit intended to prove that a product, designed without safety in mind, happens to be safe? Has a Clinical Safety Officer been involved in the detailed analysis or have they blindly added a signature at the approval stage? Is the Safety Case ‘salesy’ and full of promises or is it objective and factual?
Hazard Log – Does the hazard log contain a level of detail which is commensurate with the level of risk? At Safehand, a Hazard Log for a medium-sized clinical system might be 30-40 pages, perhaps 20 pages for a smaller app. Are the hazards a genuine assessment of how the technology could fail or are they instead a cynical attempt to transfer the risk onto users? Are the hazards wishy-washy and non-specific, e.g. Hazard: Users might not be trained, Control: Users should be properly trained.
Issues – A solid safety assessment will nearly always uncover a few issues where there is the potential to mitigate the risk further. This isn’t a failing but rather evidence of a sound and critical evaluation. A Safety Case which concludes with no concerns identified, every test passed on first attempt and all risks fully mitigated might just be portraying an overly sanitised view of the world. But conversely, don’t punish your suppliers for being honest about any shortcomings.
Method – Have the techniques used to conduct the assessment been articulated? Is there a risk matrix and defined acceptability criteria? Have the findings of the safety assessment been evaluated against the acceptability criteria? Does the scope of the assessment match what is being supplied? Without this narrative glue being incorporated into the Safety Case its findings may be considered unfounded.
Evidence – This is perhaps the most important of all; are the findings of the assessment backed up by credible evidence? In particular, where the supplier cites testing related controls, can each test execution be evidenced? A Hazard Log containing dozens of controls for which there is no evidence of implementation is simply invalid. If the supplier claims that they cannot disclose any details because of concerns over Intellectual Property then you will need to take your own view – and you would be forgiven for being sceptical.
Ultimately telling the difference between a good and bad Safety Case is a skill which we have developed in the course of reviewing the good, the bad and the ugly. But there is no room for complacency here. If you are in any doubt you should get help and this kind of critical thinking is an important part of the SCCI 0160 course we run at Safehand. Don’t be frightened to bring your supplier to task – they have a responsibility to comply with SCCI 0129 in substance and not just spirit. And when you encounter an informative, balanced and objective Safety Case, offer praise and gratitude to your supplier so they appreciate the genuine benefits and recognition of their hard work.