When BERT Plays The Lottery, All Tickets Are Winning

0
TAGS: ,

When BERT Plays The Lottery, All Tickets Are Winning

When BERT Plays The Lottery, All Tickets Are Winning

Two years after BERT was unveiled to the world, Transformers are still dominating the leaderboards and have spawned numerous follow-up studies.

The first version of our attempt to survey BERTology literature (Rogers et al. 2020) provided an overview of about 40 papers in February 2020. By June, there were over a hundred. The final TACL camera-ready version has about 150 BERT-related citations, and no illusions of completeness: we ran out of journal-allotted pages in August 2020…”

Source: thegradient.pub/when-bert-plays-the-lottery-all-tickets-are-winning/

February 24, 2021
0 Comments
Inline Feedbacks
View all comments

Subscribe to our Digest