Gekhman et al. (2024)

Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?

Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 7765-7784

Find It

DOI

Abstract

Studies what happens when a language model is fine-tuned on facts it did not acquire during pretraining, using a controlled closed-book question-answering setup in which the proportion of such facts is varied. Reports that examples introducing new knowledge are fitted substantially more slowly than examples consistent with what the model already holds, and that once the new examples are eventually fitted they linearly increase the model's tendency to answer incorrectly on facts it previously knew.

Reference

Gekhman, Z., Yona, G., Aharoni, R., Eyal, M., Feder, A., Reichart, R., & Herzig, J. (2024). Does fine-tuning LLMs on new knowledge encourage hallucinations? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (pp. 7765–7784). Association for Computational Linguistics.

What Rests on This

Cited in the argument rather than behind a score: this is one of the papers the taxonomy is built on, named where the site explains itself rather than in the findings under a mechanism.