When BERT Plays The Lottery, All Tickets Are Winning

When BERT Plays The Lottery, All Tickets Are Winning

When BERT Plays The Lottery, All Tickets Are Winning

“Two years after BERT was unveiled to the world, Transformers are still dominating the leaderboards and have spawned numerous follow-up studies.

The first version of our attempt to survey BERTology literature (Rogers et al. 2020) provided an overview of about 40 papers in February 2020. By June, there were over a hundred. The final TACL camera-ready version has about 150 BERT-related citations, and no illusions of completeness: we ran out of journal-allotted pages in August 2020…”

Source: thegradient.pub/when-bert-plays-the-lottery-all-tickets-are-winning/