ProteinGym · Substitution
Leaderboard context
Official rankings live on the ProteinGym site. This page summarizes which ProSST checkpoints to use and how they relate to the MSA-enhanced follow-up VenusREM.
Official ProteinGym board
ProSST is evaluated on the ProteinGym substitution (DMS) benchmark for zero-shot fitness prediction. Always treat proteingym.org/benchmarks as the source of truth for Spearman correlations and relative ranking — numbers there are updated by the benchmark maintainers.
Reproduce ProteinGym scoring with the repo script
zero_shot/proteingym_benchmark.py
and a matching structure-token directory (vocab size must match the checkpoint).
Structure vocabulary sizes
Released Masked LM checkpoints differ only in the structure-token vocabulary size used at quantization and pretraining. Larger vocabs give finer geometric codes; 2048 is the recommended default in the project README.
| Checkpoint | Structure vocab | Hugging Face | Notes |
|---|---|---|---|
| ProSST-20 | 20 | AI4Protein/ProSST-20 | Coarse structure codes |
| ProSST-128 | 128 | AI4Protein/ProSST-128 | — |
| ProSST-512 | 512 | AI4Protein/ProSST-512 | — |
| ProSST-1024 | 1024 | AI4Protein/ProSST-1024 | — |
| ProSST-2048 | 2048 | AI4Protein/ProSST-2048 | Recommended default |
| ProSST-4096 | 4096 | AI4Protein/ProSST-4096 | Finest released vocab |
Relation to VenusREM
VenusREM is an MSA-enhanced follow-up that builds on ProSST’s structure-aware modeling. On ProteinGym’s Substitution track it has been reported at approximately 0.518 Spearman’s ρ in the ProSST README project news (approximate / as stated there — confirm against the live ProteinGym board).
In short: use ProSST when you want a structure-token Masked LM without MSA retrieval; use VenusREM when MSA retrieval is available and you care about top Substitution-benchmark performance. Both are integrated into VenusFactory2.