noun
A transformer is a neural network architecture designed to process sequential data, such as text or audio, by using self-attention mechanisms to weigh the importance of different parts of the input.
It was introduced in 2017 as a breakthrough in natural language processing, replacing older models like recurrent neural networks (RNNs) by handling long-range dependencies more efficiently.
For developers, transformers enable powerful applications like real-time translation or image generation, but they require significant computational resources and training data. For example, training a large transformer model like GPT-3 can cost over $1 million in cloud computing fees.
A transformer model can take a sentence in English and generate a corresponding image, like turning 'a red cat sleeping on a couch' into a realistic picture.
In other languages
See also: Neural network · Attention mechanism