Algorithmic bias
Systematic error in an AI tool that falls unevenly across groups of patients, so the tool works less well for some people than its overall numbers suggest. It enters at several points. A group can be thin in the training data. An outcome can be recorded less often or less accurately for some patients, which happens when a test is ordered less often in an underserved population. The label itself can be a proxy, such as past healthcare spending standing in for health need, so the model learns who received care and treats that as who needed it. A single overall accuracy figure hides all of this. Before a tool goes live, ask for performance broken down by the groups your service treats: sex, age, ethnicity, language, and the sites and scanners you run.
A skin lesion classifier reports high accuracy in its validation paper. Most of the images came from patients with light skin, and the paper gives no breakdown by skin type. Used in a clinic with a broad mix of patients, it misses more melanomas on darker skin, and no dashboard flags it, because the overall rate looks fine and the misses are spread thin. The fix starts before purchase: ask for results by skin type, and where the vendor has none, treat the tool as unvalidated for the patients it lacks.
Terms like this come up in real clinical scenarios across the HelloAI courses: bite-sized modules with verifiable certificates. An account takes one minute, no password needed.
Sign in →