AI and machine learning technologies are expected to bring numerous changes to the healthcare sector, but their complexity confronts users, developers and regulators alike with unique challenges and considerations.

To help lay the groundwork for the safe and effective development of medical devices that use AI and ML, the U.S. Food and Drug Administration, along with its U.K. and Canadian counterparts, recently released a list of "guiding principles” to inform what has been dubbed “Good Machine Learning Practice (GMLP).”

In a statement outlining the principles, the FDA said it and its allies “envision these guiding principles may be used to:

    •    Adopt good practices that have been proven in other sectors

    •    Tailor practices from other sectors so they are applicable to medical technology and the health care sector

    •    Create new practices specific for medical technology and the health care sector”

The principles are:  

    1.    The total product life cycle uses multidisciplinary expertise: “In-depth understanding of a model’s intended integration into clinical workflow . . . can help ensure that ML-enabled medical devices are safe and effective and address clinically meaningful needs over the lifecycle of the device.”

    2.    The model design is implemented with good software engineering and security practices: “These practices include methodical risk management and design process that can appropriately capture and communicate design, implementation, and risk management decisions and rationale, as well as ensure data authenticity and integrity.”

    3.    Participants and data sets represent the intended patient population: “This is important to manage any bias, promote appropriate and generalizable performance across the intended patient population, assess usability, and identify circumstances where the model may underperform.”

    4.    Training data sets are independent of test sets: “All potential sources of dependence, including patient, data acquisition, and site factors, are considered and addressed to assure independence.”

    5.    Selected reference data sets are based upon best available methods: “Accepted, best available methods for developing a reference dataset (that is, a reference standard) ensure that clinically relevant and well characterized data are collected and the limitations of the reference are understood.”

    6.    Model design is tailored to the available data and reflects intended device use: “The clinical benefits and risks related to the product are well understood, used to derive clinically meaningful performance goals for testing, and support that the product can safely and effectively achieve its intended use.”

    7.    Focus is placed on the performance of the human-AI team: “(H)uman factors considerations and the human interpretability of the model outputs are addressed with emphasis on the performance of the Human-AI team, rather than just the performance of the model in isolation.”

    8.    Testing demonstrates device performance during clinically relevant conditions: “Considerations include the intended patient population, important subgroups, clinical environment and use by the Human-AI team, measurement inputs, and potential confounding factors.”

    9.    Users are provided clear, essential information: “Users are also made aware of device modifications and updates from real-world performance monitoring, the basis for decision-making when available, and a means to communicate product concerns to the developer.”

    10.    Deployed models are monitored for performance, and retraining risks are managed: “Deployed models have the capability to be monitored in “real world” use with a focus on maintained or improved safety and performance.”

The agency says stakeholders can use the principles to tailor and adopt good practices from other sectors to be used in the health tech sector, as well as to create new specific practices.  

"Areas of [international] collaboration include research, creating educational tools and resources, international harmonization, and consensus standards, which may help inform regulatory policies and regulatory guidelines," said FDA officials.

Photo by AzriSuratmin/Getty Images