PaperNote (Total 10 articles)
NIPS-2021 Align before Fuse：Vision and Language Representation Learning with Momentum Distillation
Bioinformatics-2021 DNABERT：pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome
ECCV-2022 Contrastive Deep Supervision
OpenAI-2022 Robust Speech Recognition via Large-Scale Weak Supervision
TPAMI-2020 ProtTrans：Towards Cracking the Language of Life’s Code Through Self-Supervised Learning
Nature Machine Intelligence-2022 Learning functional properties of proteins with language models
ICLR-2021 Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning
JMLR-2022 Switch Transformers：Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
NIPS-2019 Unified Language Model Pre-training for Natural Language Understanding and Generation
ICLR-2022 OntoProtein：Protein Pretraining With Gene Ontology Embedding