# SparK✨: the first successful BERT-style pre-training on any convolutional networks [](https://arxiv.org/abs/2301.03580), ICLR'23 Spotlight
# SparK✨: the first successful BERT-style pre-training on any convolutional networks [](https://arxiv.org/abs/2301.03580) [ICLR'23 Spotlight]
Official implementation of "Designing BERT for Convolutional Networks: ***Spar***se and Hierarchical Mas***k***ed Modeling".