kandpal-2023-cooccurrence-over-entity-presence
IN premise — summaries/2026/08/24/kandpal-2023-long-tail-knowledge-sR-references.md
Created 2026-08-25T02:58:07+00:00
Counting documents where both the question entity and answer entity co-occur is the valid predictor of QA accuracy, whereas counting documents containing only one entity is confounded by entity popularity and loses predictive power when restricted to QA pairs with fewer than 5 co-occurrences.
Summary
To judge whether a QA system truly knows a question-answer connection, you must count documents where both entities appear together, not just documents mentioning one of them. This matters because simply counting one entity's appearances mostly reflects how popular that entity is, which becomes useless as a signal exactly in the low-frequency range where you most need to distinguish real knowledge from noise.