Cross-Attention
Cross-Attention is a transformer mechanism where queries from one sequence attend to keys and values from a different source sequence, enabling models to integrate contextual information across modalities or sequences, such as decoder states querying encoder outputs.
Cross-Attention is a transformer mechanism where queries from one sequence attend to keys and values from a different source sequence, enabling models to integrate contextual information across modalities or sequences, such as decoder states querying encoder outputs.