ICML-2023 Structure-informed Language Models Are Protein Designers

2023-02-09 · ICML· Biology · protein, PLM, protein-design

Paper: Structure-informed Language Models Are Protein Designers

Code: https://github.com/BytedProtein/ByProt

LM-DESIGN: Structure models prompt sequence models to refine protein design sequences

Abstract

A general approach built on sequence-based protein language models (PLMs) inserts lightweight structural adapters into PLMs and applies iterative refinement during inference to effectively optimize generated protein sequences.

Introduction

Structure-based protein design is mainly limited by:

Limited experimentally determined protein structure data
Regions of structural uncertainty

From a biological perspective, protein structure sometimes carries insufficient information, especially for flexible regions such as loops and exposed surfaces. In these regions, residue identity is assumed to correlate less with structural context, whereas sequence knowledge is more useful but has largely been ignored. The authors validate this hypothesis and find that existing purely structure-based methods tend to produce functionally invalid sequences for such regions.

Therefore, structure models are used to prompt sequence models to generate sequences.

Reprogramming pLMs for Structure-based Protein Design with Structure Surgery

Adding this structural adapter is a model-agnostic approach.

Training

Denoising autoencoding with conditional masked language modeling (CMLM); for sequence generation, this objective—exposing the model only to amino acids on the left—is preferable.

The initial sequence comes from mapping the structure model to the 20 amino acids via a linear head; the method iteratively refines for T steps until optimization plateaus.