popqa-accuracy-substring-match

Summary

POPQA scores count a model's answer as correct even if the right option just appears as a fragment buried inside a longer or messier generation, not only when the model produces the exact option on its own. This means POPQA accuracy numbers tend to flatter a model's performance, so when comparing models or tracking progress, you should treat those scores as a generous floor rather than a tight measure of what the model actually "knows."