NIPS-2021 VLMo：Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts

Dec 14 PaperNote CL, CV, cs.ContraLern, cs.Multimodal, cs.Transformer Comments Word Count: 466(words) Read Count: 1(minutes)

NIPS-2021 Align before Fuse：Vision and Language Representation Learning with Momentum Distillation

Dec 12 PaperNote CL, CV, cs.ContraLern, cs.Multimodal, cs.Transformer Comments Word Count: 1.2k(words) Read Count: 4(minutes)

DNA Replication and Repair

Dec 9 MIT-molecular biology Comments Word Count: 709(words) Read Count: 2(minutes)

Bioinformatics-2021 DNABERT：pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome

Dec 9 PaperNote BI, cs.Transformer Comments Word Count: 1.2k(words) Read Count: 4(minutes)

ECCV-2022 Contrastive Deep Supervision

Nov 22 PaperNote CV, cs.ContraLern Comments Word Count: 1.2k(words) Read Count: 4(minutes)

OpenAI-2022 Robust Speech Recognition via Large-Scale Weak Supervision

Nov 19 PaperNote cs.Speech, cs.Transformer Comments Word Count: 1.8k(words) Read Count: 6(minutes)

TPAMI-2020 ProtTrans：Towards Cracking the Language of Life’s Code Through Self-Supervised Learning

Nov 17 PaperNote BI, cs.Transformer Comments Word Count: 1.4k(words) Read Count: 5(minutes)

Nature Machine Intelligence-2022 Learning functional properties of proteins with language models

Nov 16 PaperNote BI Comments Word Count: 4.1k(words) Read Count: 14(minutes)

ICLR-2021 Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning

Nov 13 PaperNote CL, cs.ContraLern, cs.Transformer Comments Word Count: 720(words) Read Count: 2(minutes)

JMLR-2022 Switch Transformers：Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

Nov 12 PaperNote CL, cs.Transformer Comments Word Count: 2.7k(words) Read Count: 9(minutes)