BERT
A well-known encoder-only (autoencoding) transformer, pre-trained with masked language modeling so it reads context bidirectionally. Used for understanding tasks like classification. In healthcare, the BEHRT adaptation was trained on sequential diagnosis codes to predict future diagnoses.
A quality team wants to catch every radiology report that recommends follow-up imaging, and the informatics group fine-tunes a BERT-style model to flag them. Reading each phrase in light of the words on both sides, it copes with wording such as "no further imaging needed" better than a keyword search. Standard BERT accepts at most 512 tokens, a few hundred words, so in a long CT report the impression, where the recommendation often sits, can be cut off. Ask how long reports are handled, and measure missed recommendations on your own longest reports.
Terms like this come up in real clinical scenarios across the HelloAI courses: bite-sized modules with verifiable certificates. An account takes one minute, no password needed.
Sign in →