Question: can we find a good context permutation to improve reasoning capabilities.
One-Liner
Notable Methods
Two key evaluations:
- evalutanig relationships between gold documents; notice that performance relates to distance between documents (but FTing helps)
- investigate the effects between different attention masks (i.e., the use of prefix vs continuation masks)
IC Score
attention-based context attribution method
New Concepts
Key insight: correct answers will have single peak of IC scores at gold results; incorrect answers will have more dispersed IC scores.
=>
relevant documents should be placed next to each other