Misalignment audits

KeyVerifiedUnverifiedDisputedCorrected* read off a figure or stated in wordsA value opens its source and history.

• Each of the 24 evaluations is its own test, with its own tasks, graders and definitions. Values from different evaluations describe the same property measured in different ways, and are not comparable with one another, even where two evaluations share a name.
• Values come from the documents of four developers. Developers use different tests, prompts and graders for the same property, so we do not rank them on these numbers: tables here are ordered by name or by kind of test, never by a reported value.
• In three evaluations the test changed between documents (Destructive action avoidance, Automated behavioral audit and Frontier Safety Framework determination (misalignment)). Values on the old and new definitions are never joined: the explorer breaks the line where the test changed.
• In three evaluations the explorer connects results from different documents that do not say whether the test stayed the same. We connect them because nothing suggests it changed; where a document says it did, the line breaks. Each value on such a line is marked “test assumed unchanged”.
• Nine evaluations report values under more than one condition, such as reasoning effort, safeguards or number of attempts. Each condition is kept as a separate series.
• Numbers in this family are printed in five units: percent, counts, relative change in percent, rates from 0 to 1 and scores from 1 to 10. We never convert one unit into another, so a chart shows one unit at a time.
• For some measures a higher value is worse and for others better, depending on whether the developer reports the behaviour or its absence. Each value’s record says which.
• Four values are restatements: a later document reporting an earlier model again, sometimes with a different value. A restatement is kept beside the model’s own value, never in place of it.

No finding is about this family yet.