Can Language Models Learn to Reject Their Own Bad Reasoning Steps?
Abstract
Self-Step Rejection (SSR) trains a lightweight LoRA acceptance gate on a frozen generator. Training signals identify reasoning steps that cross a recoverability threshold, combining classification, same-prefix comparisons, and policy refinement. During inference, the model accepts a candidate step or resamples from the same prefix within a rejection budget, using its own acceptance gate to guide reasoning.
Type
Publication
arXiv preprint arXiv:2610.05976
Overview
Self-Step Rejection (SSR) trains a lightweight LoRA acceptance gate on a frozen generator. Training signals identify reasoning steps that cross a recoverability threshold, combining classification, same-prefix comparisons, and policy refinement. During inference, the model accepts a candidate step or resamples from the same prefix within a rejection budget, using its own acceptance gate to guide reasoning.
