kandpal-2023-pipeline-precision-60pct

IN premise — summaries/2026/08/24/kandpal-2023-long-tail-knowledge-s2-identifying-relevant-pre-training-data.md

Created 2026-08-25T02:58:06+00:00

The entity-linking pipeline achieved approximately 60% precision when human-evaluated on TriviaQA, with ~33% of sampled relevant documents containing enough information to answer and ~27% containing partial relevance.

Summary

This is a measured snapshot showing the entity-linking stage gets roughly two out of every five links right, and a big reason for that is that most retrieved documents either lack a full answer or only give a fragment of one. For the system, it means downstream reasoning is working from a noisy foundation, so any accuracy gains will need to come from either better linking or stricter document filtering before the answer step.