Abstract. The increasingly widespread adoption of large language models has highlighted the need for improving their explainability. We present context length probing, a novel explanation technique for causal language models, based on tracking the predictions of a model as a function of the length of available context, and allowing to assign differential importance scores to different contexts. The technique is model-agnostic and does not rely on access to model internals beyond computing token-level probabilities. We apply context length probing to large pre-trained language models and offer some initial analyses and insights, including the potential for studying long-range dependencies.
We provide two demos: a static one with the models and the dataset from the paper, and an interactive one (a 🤗 Space) for trying out some smaller models on arbitrary inputs.
Hover over a token to see the left-hand context highlighted according to importance scores. The score expresses the reduction in the selected metric (green: decrease, red: increase) caused by adding a given token to the context. Click a token to pin it, then try hovering over the context to see the corresponding top 5 predictions for the pinned token (enable Show top predictions first). Click anywhere to unpin the token.
The text is from the UD_English_LinES treebank from Universal Dependencies, distributed under the CC BY-NC-SA 4.0 license.