Attention Visualization Between Sentences

What is Attention?

Attention is a mechanism that allows models to focus on different parts of the input when producing output. In the context of natural language processing, attention helps determine how much focus should be placed on specific words or sentences when processing text.

How This Visualization Works

This tool splits your text into sentences and computes attention scores between each pair of sentences. The attention is calculated using vector representations of the sentences and the selected attention method.

Interpreting the Visualization

1. Each cell in the heatmap represents the attention score between a pair of sentences.
2. Darker red cells indicate higher attention (stronger relationship).
3. Click on any cell to see detailed information about the relationship between those sentences.
4. The insights section provides an automated analysis of the attention patterns.

Available Attention Types

• Cosine Similarity: Measures the cosine of the angle between two sentence vectors (value from -1 to 1, normalized to 0-1).
• Dot Product: Calculates the product of corresponding elements of two sentence vectors.
• Scaled Dot Product: Dot product scaled by the square root of the vector dimension (similar to attention in transformer models).