Beyond Magic Words: Sharpness-Aware Prompt Evolving for Robust Large Language Models
Apr 23, 2026·,,
,,,,,,,,,·
1 min read
Guancheng Wan
Lucheng Fu
Haoxin Liu
Yiqiao Jin
Hejia Geng
Eric Hanchen Jiang
Hui Yi Leong
Jinhe Bi
Yunpu Ma
Xiangru Tang
B. Aditya Prakash
Yizhou Sun
Wei Wang
Abstract
Prompt engineering for large language models often relies on brittle ‘magic words’ that fail to generalize across tasks and models. We propose a sharpness-aware prompt evolving framework that optimizes prompts for robustness by penalizing sharp regions in the prompt loss landscape. Across diverse benchmarks and LLM families, sharpness-aware prompt evolution produces prompts that are both stronger on the target task and more transferable. ICLR 2026 acceptance rate: 28%.
Type
Publication
International Conference on Learning Representations (ICLR) 2026
Abstract
Prompt engineering for large language models often relies on brittle “magic words” that fail to generalize across tasks and models. We propose a sharpness-aware prompt evolving framework that optimizes prompts for robustness by penalizing sharp regions in the prompt loss landscape.
