Research on fine-tuning algorithms for Large Language Models integrating Uncertainty Modeling and External Memory Augmentation

by Yumeng Ma, Yue Xing, Di Wu, Yining Zhou, Yun Zi, Ming Wang, Yingnan Deng, Shuaidong Pan

This paper proposes a parameter-efficient fine-tuning framework that integrates uncertainty modeling with external memory augmentation, aiming to improve robustness, confidence calibration, and contextual completeness in downstream natural language processing tasks. From the methodological perspective, the uncertainty modeling module explicitly characterizes uncertainty in inputs and intermediate representations through feature-level estimation, cross-layer propagation, and confidence calibration, thereby enhancing training stability and reducing the influence of noisy signals. Meanwhile, the external memory augmentation module employs key-value retrieval and gated fusion mechanisms to provide reusable contextual support, alleviating information loss caused by limited contextual summarization and improving representation quality under heterogeneous evaluation settings. Extensive experiments and ablation studies were conducted on text classification and named entity recognition tasks across multiple public benchmark datasets, using GPT-2 Small, GPT-2 Medium, and LLaMA3-8B as backbone models. The results demonstrate that the proposed framework consistently outperforms several mainstream fine-tuning methods in terms of accuracy, F1 score, and robustness, while also showing stable behavior under learning-rate sensitivity and missing-information settings. Overall, this study provides a novel perspective for efficient and interpretable fine-tuning paradigms, achieving a favorable balance among performance improvement, parameter efficiency, and deployment feasibility, and offering a practical basis for future extensions to more complex downstream scenarios.