Red teaming
Deliberately attacking or stress-testing an AI system to find where it fails before it is deployed, by trying to make it produce harmful, wrong, or out-of-bounds output. The name comes from security, where a "red team" plays the adversary. It is becoming a standard expectation in AI safety and is now appearing in procurement and governance conversations. For decision-makers it is a fair question to ask of any healthcare AI vendor: was this system red-teamed, by whom, and what did they find, because a tool that has never been tested against misuse has unknown failure modes.
Before a patient-facing chatbot for medication and appointment questions goes live, a hospital asks a pharmacist, two nurses and a resident from outside the project to spend a week trying to make it answer unsafely. They get it to suggest doubling a missed blood thinner dose when the question is phrased loosely, and to offer self-care tips for chest pain described in vague terms. Fix those, keep the test questions as a standing set, and rerun them after every model or prompt update.
Terms like this come up in real clinical scenarios across the HelloAI courses: bite-sized modules with verifiable certificates. An account takes one minute, no password needed.
Sign in →