Glossary

Multi-Head Attention

**Multi-Head Attention (MHA)** is a core Transformer mechanism that performs multiple parallel attention computations across distinct 'heads' to jointly capture diverse relational patterns, such as syntactic dependencies, semantic similarities, and positional relationships, in sequential data.

**Multi-Head Attention (MHA)** is a core Transformer mechanism that performs multiple parallel attention computations across distinct 'heads' to jointly capture diverse relational patterns, such as syntactic dependencies, semantic similarities, and positional relationships, in sequential data.