While the idea of AI in healthcare typically refers to the role emerging technologies are playing in both disease diagnosis on the front end and administrative process on the back end, few would dispute the is that preventing child abuse or neglect is, on one level, very much a worthy challenge for the healthcare sector.
To that end, child protective services agencies have increasingly been implementing machine learning models to help child welfare specialists screen cases to determine which merit further investigation. To help improve the use of ML in such screening, a team led by researchers from MIT has partnered with a child welfare department in Colorado to study how call screeners assess cases.
According to a statement describing the study, “(b)ased on feedback from the call screeners, they designed a visual analytics tool that uses bar graphs to show how specific factors of a case contribute to the predicted risk that a child will be removed from their home within two years.
The researchers found that screeners are more interested in seeing how each factor, like the child’s age, influences a prediction, rather than understanding the computational basis of how the model works. Their results also show that even a simple model can cause confusion if its features are not described with straightforward language.”
Senior author Kalyan Veeramachaneni, principal research scientist in the Laboratory for Information and Decision Systems (LIDS) and senior author of the paper, noted that “domain experts (the screeners) don’t necessarily want to learn what machine learning actually does. They are more interested in understanding why the model is making a different prediction than what their intuition is saying, or what factors it is using to make this prediction. They want information that helps them reconcile their agreements or disagreements with the model, or confirms their intuition.”
Based on observations and detailed interviews, the researchers built four additional interfaces, including one that compares a current case to past cases with similar risk scores. Subsequent user studies revealed that more than 90 percent of the screeners found the case-specific details interface to be useful, and it generally increased their trust in the model’s predictions. On the other hand, the screeners did not like the case comparison interface. While the researchers thought this interface would increase trust in the model, screeners were concerned it could lead to decisions based on past cases rather than the current report.
Moving forward, they plan to enhance the interfaces they’ve created based on additional feedback and then run user studies to track the effects on decision making with real cases, all with the goal of facilitating more targeted assessment of abuse cases.
Photo by monsitj/Getty Images