popqa-16-relationship-types

IN premise — summaries/2026/08/24/mallen-2023-when-not-to-trust-s3-evaluation-setup.md

Created 2026-08-25T02:58:11+00:00

POPQA samples knowledge triples across 16 relationship types from Wikidata.

Summary

POPQA is designed to test a broad range of how entities relate to each other, drawing from 16 different relationship categories in Wikidata rather than a single narrow type of fact. This means performance on the benchmark reflects general relational knowledge, and any system or model being evaluated there is being probed across many diverse question patterns at once.