ripple-edits-nrel-54-relations

IN premise — summaries/2026/08/24/cohen-2023-ripple-effects-s4-the-r-ipple-edits-benchmark.md

Created 2026-08-25T02:57:56+00:00

The RIPPLE EDITS benchmark uses Nrel = 54 basic relations for which manually crafted natural-language templates and logical rules are defined.

Summary

The RIPPLE EDITS benchmark is scoped to a fixed set of 54 relationship types, each backed by hand-written language templates and logic rules, so any system evaluated on it is being measured against a constrained, human-defined vocabulary rather than open-ended relationships. In practice, this means results from the benchmark only tell you how well a model handles these specific 54 patterns and say nothing about its broader relational reasoning.