Softmax
A function that turns a set of raw scores into a probability distribution that sums to one. Transformers use it inside attention to turn comparison scores into weights, and at the final layer to turn vocabulary scores into the probability of each possible next word.
A dermatology triage model trained to choose among six lesion types is shown a photo of something outside that list, such as an insect bite. Its final softmax layer shares out the full probability among the six classes it knows, so it still names one of them, sometimes with a high score. Read the score as confidence relative to those six classes only. Before using it where unexpected cases arrive, ask the vendor what the product does with images outside its label set and whether it has a reject or out-of-scope option.
Terms like this come up in real clinical scenarios across the HelloAI courses: bite-sized modules with verifiable certificates. An account takes one minute, no password needed.
Sign in →