Jailbreak robustness

KeyVerifiedUnverifiedDisputedCorrected* read off a figure or stated in wordsA value opens its source and history.

• Each of the 16 evaluations is its own test, with its own tasks, graders and definitions. Values from different evaluations describe the same property measured in different ways, and are not comparable with one another, even where two evaluations share a name.
• Values come from the documents of six developers. Developers use different tests, prompts and graders for the same property, so we do not rank them on these numbers: tables here are ordered by name or by kind of test, never by a reported value.
• Some names are used by more than one evaluation: “StrongReject” by four and “Jailbreaks” by two. They are different tests, run by different evaluators or reported by different developers, so this page names who ran each one.
• In four evaluations the explorer connects results from different documents that do not say whether the test stayed the same. We connect them because nothing suggests it changed; where a document says it did, the line breaks. Each value on such a line is marked “test assumed unchanged”.
• Three evaluations report values under more than one condition, such as reasoning effort, safeguards or number of attempts. Each condition is kept as a separate series.
• Numbers in this family are printed in two units: rates from 0 to 1 and percent. We never convert one unit into another, so a chart shows one unit at a time.
• For some measures a higher value is worse and for others better, depending on whether the developer reports the behaviour or its absence. Each value’s record says which.
• Two values are restatements: a later document reporting an earlier model again, sometimes with a different value. A restatement is kept beside the model’s own value, never in place of it.

No finding is about this family yet.