Screening Is Enough

1. 1
   Grasp Softmax's Relative Nature

   Recognize that softmax attention weights are always normalized to sum to 1.0, meaning the 'importance' of any key is always defined in comparison to all other keys present in the context, not by its standalone score.
2. 2
   Observe Contextual Impact on Weights

   Run the provided starter code to see how adding or removing keys, or changing their scores, directly affects the attention weights of *all* other keys, even if their raw scores remain unchanged. This demonstrates the 'fixed unit mass' distribution.
3. 3
   Adapt Model Interpretation and Debugging

   When analyzing attention maps or debugging model behavior, always consider the full set of keys being attended to. A key's high attention weight might be due to its relative dominance in a weak context, not necessarily its absolute significance.
4. 4
   Consider Absolute Relevance Alternatives

   For applications requiring precise, absolute relevance assessments, explore alternative attention mechanisms or model architectures that can score keys independently, rather than relying solely on softmax's relative distribution.

import numpy as np

def softmax(x):
    e_x = np.exp(x - np.max(x)) # Subtract max for numerical stability
    return e_x / e_x.sum()

print("\n--- Scenario 1: Two keys ---")
scores_1 = np.array([2.0, 1.0]) # Key 1 is stronger
attention_weights_1 = softmax(scores_1)
print(f"Scores: {scores_1} -> Weights: {np.round(attention_weights_1, 3)}")

print("\n--- Scenario 2: Add a less relevant third key ---")
# Key 1 & 2 raw scores unchanged, but total mass is now split among 3
scores_2 = np.array([2.0, 1.0, 0.5]) 
attention_weights_2 = softmax(scores_2)
print(f"Scores: {scores_2} -> Weights: {np.round(attention_weights_2, 3)}")
print("Notice how weights for Key 1 and Key 2 decreased, even though their raw scores didn't change. This is relative relevance.")

print("\n--- Scenario 3: All keys equally important (high scores) ---")
scores_3 = np.array([10.0, 10.0, 10.0])
attention_weights_3 = softmax(scores_3)
print(f"Scores: {scores_3} -> Weights: {np.round(attention_weights_3, 3)}")
print("Fixed unit mass distributed equally when scores are identical.")

• Paperarxiv.org