Many people have turned to AI with a well being query or obtained care from a physician who’s utilizing it. The adoption of know-how for well being is accelerating and impacts billions of individuals day-after-day. However how can we assess if these instruments are secure and useful?
In partnership with well being consultants Dr. Elizabeth McGlynn and Dr. Dana Gelb Safran, we published a whitepaper detailing an end-to-end analysis method to evaluate AI use instances for addressing folks’s well being wants. The framework evaluates fashions throughout three areas: accuracy, security, and usefulness. It combines closed-question benchmarks, autoraters, and knowledgeable human evaluate to evaluate widespread and high-risk well being eventualities.
We constructed and proceed to make the most of this framework to evaluate whether or not our fashions present useful, accurate responses that promote safe interactions in well being contexts. By sharing our methodology, we hope to advertise transparency in how we method this vital space, and search to assist all the well being ecosystem — from builders to researchers to clinicians — in constructing AI instruments folks can belief.
