Your model already knows the answer: how benchmark answers leak into LLMs

AI Summary

Large language models (LLMs) can inadvertently incorporate benchmark answers into their training data, leading to predictable responses. This phenomenon raises questions about the integrity of model evaluations and the reliance on benchmark datasets.