During security testing of a foundation model, testers intentionally bypass the model's safety protections to produce harmful content. What is this kind of technique called?

Jailbreaking refers to techniques used to bypass the safety and ethical guidelines of a foundation model, often to elicit harmful, biased, or restricted content. Testers intentionally craft prompts or inputs to circumvent the model's intended guardrails. Fuzzing training data involves providing malformed or unexpected inputs to a system to uncover vulnerabilities, but it's typically applied to the data itself rather than directly bypassing model safety. Denial of Service (DoS) attacks aim to make a system unavailable to its intended users, which is a different security concern. Authorized penetration testing is a broad term for security testing conducted with permission, but "jailbreak" specifically describes the technique of bypassing model safety features.