Explainability
How far a person can understand why an AI tool produced a particular output. There are two routes to it. An interpretable model is simple enough to read directly, like a points-based risk score where you can see what each factor added. An explanation method sits on top of a complex model and approximates its reasoning after the fact, and the heatmap over a chest X-ray is the familiar example. The two give different things. A heatmap shows which pixels most influenced the output, and it says nothing about whether the finding there is real or why the model weighed it. Explanations can also make a wrong output look more convincing. For deciding whether to trust a tool, validation on your own patients tells you more than an explanation of a single case.
A fracture tool marks a wrist X-ray positive and paints a heatmap over the distal radius. The heatmap shows which pixels moved the score, and those might be the fracture line or might be the cast, the ruler, or a marker near that spot. Read the heatmap as a pointer to where you should look, then judge the finding yourself. If a tool's heatmaps keep landing on things that are not anatomy, that tells you something about what it learned.
Terms like this come up in real clinical scenarios across the HelloAI courses: bite-sized modules with verifiable certificates. An account takes one minute, no password needed.
Sign in →