How the January jobs report could impact the Fed's monetary policy

Proposes a new deep learning model called Transformer that relies entirely on attention mechanisms without recurrence or convolution.

Transformer achieves state-of-the-art results on multiple machine translation tasks like WMT 2014 English-to-German and WMT 2014 English-to-French.

The Transformer model is the first transduction model relying entirely on self-attention to compute representations of its input and output without using sequence-aligned RNNs or convolution.

Transformer uses stacked self-attention and point-wise, fully connected layers for both encoder and decoder.

Multi-head attention allows model to jointly attend to information from different representation subspaces at different positions.

Positional encodings provide Transformer's sequential representation ability without recurrence.

Transformer achieves superior quality in translation task compared to best results on WMT 2014 English-to-German using convolutions or LSTMs.

Self-attention could provide fastertraining and inference speed over RNNs and CNNs for seq2seq tasks.

Transformer is more parallelizable than RNNs, requiring significantly less time to train for translation task.

Attention-based models can propagate information much more quickly than RNNs,since ints do not need to process sequentially.

Self-attention could enable modeling of longer range dependencies in data than RNNs.

Transformer is a major contribution demonstrating effectiveness of attention and replacing recurrence for translation.

Transformer architecture has become pervasive, achieving SOTA across NLP tasks like question answering, summarization.

Transformer was a breakthrough in demonstrating power of attention mechanisms and remains highly influential in NLP.

Transformer has inspired countless subsequent models leveraging self-attention, being foundational technique.