Tom on Twitter
ELMo’s LM was bi-directional, but the openAI transformer only trains a forward LM. A transformer-based model whose LM looks both forward & backwards?-> transformer encoders :: BERT https://t.co/j20F9RFoD5READ:https://t.co/8OsE3c7Vxh#LM_M = Language Model, masked #NLP— Tom (@data_topology) December 14, 2018