In 2017 researchers at Google published the transformer, a neural network design that handles language well and can be trained on very large amounts of text.
Later models, such as OpenAI's GPT series and Google's BERT, were trained at larger and larger scales. ChatGPT, released in November 2022, brought large language models to a mass audience.