Self-supervised learning
Training a model on large amounts of unlabelled data by having it predict parts of its own input, so no expensive expert annotations are needed. It is the usual first stage for transformers, producing general representations that a later fine-tuning step adapts to a specific task.
A hospital research group wants a model that grades knee osteoarthritis on X-ray, but its radiologists have time to label only a few hundred films. The archive holds hundreds of thousands of unlabeled knee images, so the team first trains the model to reconstruct hidden patches of those images, then fine-tunes it on the small labeled set. What still needs radiologist time is a carefully labeled test set from patients the model never saw in either stage, so budget for that first, since it is where accuracy gets measured.
Terms like this come up in real clinical scenarios across the HelloAI courses: bite-sized modules with verifiable certificates. An account takes one minute, no password needed.
Sign in →