Smaller, faster, cheaper, lighter: Introducing DilBERT, a distilled version of BERT

Smaller, faster, cheaper, lighter: Introducing DilBERT, a distilled version of BERT

🏎 Smaller, faster, cheaper, lighter: Introducing DilBERT, a distilled version of BERT

You can find the code to reproduce the training of DilBERT along with pre-trained weights for DilBERT here.

Source: medium.com/huggingface/distilbert-8cf3380435b5