Vision transformer
A transformer applied to images rather than text, used as an alternative to convolutional networks (CNNs). It splits an image into patches and uses attention to relate them, borrowing the transformer advantages of scaling to large data and training end to end without hand-designed layers. Vision transformers are increasingly common in radiology and pathology AI.
A vendor's mammography model uses a vision transformer, which cuts the image into small square patches and compares each with every other. Doubling the width and height means four times the patches and about sixteen times the comparisons, so developers either downsample, which can blur fine detail such as microcalcifications, or tile the image, which keeps detail but means no single pass sees the whole breast. Ask which approach the product uses and at what resolution, and check its performance on cancers that present only as calcifications.
Terms like this come up in real clinical scenarios across the HelloAI courses: bite-sized modules with verifiable certificates. An account takes one minute, no password needed.
Sign in →