Encoder-only model
A transformer that uses only the encoder, also called an autoencoding model. Built for understanding context, such as classification and sentiment analysis. BERT is the classic example, pre-trained with masked language modeling and fine-tuned by adding classifier layers.
A documentation tool flags progress notes that suggest sepsis, using a classifier built on a model that reads each note in both directions at once. Many models of this family were built to accept at most 512 tokens, a few hundred words, and a longer note is either cut off or split into pieces that are scored separately. The assessment and plan usually come last, so truncation can drop the part that matters most. Ask what the input limit is and how longer notes are handled, then test on long notes where the key finding sits at the end.
Terms like this come up in real clinical scenarios across the HelloAI courses: bite-sized modules with verifiable certificates. An account takes one minute, no password needed.
Sign in →