Inferring from Logits: Exploring Best Practices for Decoding-Free Generative Candidate Selection
Existing works have been using decoding-free candidate selection methods to obtain candidate probability from initial output logits over vocabulary. Though these estimation methods are widely used, they are not systematically evaluated, especially on end tasks. We introduce an evaluation of a comprehensive collection of decoding-free candidate selection approaches.
Oct 3, 2024