AI Mixologist vs. Human Bartender: Can You Taste the Difference? | WIRED

Proposes a new deep learning model called BERT that achieves state-of-the-art performance on 11 NLP tasks

BERT is the first model to deeply bidirectional pre-train natural language representations using masked language modeling

The pre-trained BERT model can be fine-tuned with just one additional output layer for state-of-the-art performance on tasks like question answering and language inference

BERT large model achieves GLUE score of 80.5%, outperforming previous state-of-the-art models by a large margin

BERT base model achieves state-of-the-art on SQuAD 1.1 question answering by 3.5 F1 score

Masked language modeling and next sentence prediction tasks were key to BERT's bidirectional pre-training success

BERT has fewer parameters than previous models like GPT, allowing more efficient fine-tuning on tasks

BERT represents words as WordPiece embeddings with 30,000 tokens

BERT base model has 12 transformer blocks, 12 attention heads, and 110 million parameters

BERT large model has 24 transformer blocks, 16 attention heads, and 340 million parameters

Training the BERT large model took 4 days on 4 to 16 Cloud TPUs

BERT shows substantial gains across a diverse range of NLP tasks including sentiment analysis, paraphrasing, and entailment

BERT opens up many possibilities for transfer learning in NLP and advancing the state-of-the-art across tasks

Code and pre-trained models are made available to facilitate research into new fine-tuning approaches for BERT

BERT is a major breakthrough demonstrating the power of pre-training contextual representations for NLP