Attention mechanism

A mathematical operation developed by Dzmitry Bahdanau in 2014 that enables neural networks to dynamically focus on relevant parts of input data, such as words in a sentence, by assigning weights to contextual relationships. It became the core of transformer architectures and large language models.

Catégorie : Sciences