DSM¶

Masked diffusion protein language model for sequence generation (unconditional, masked infilling, conditional), embedding extraction, and log-probability scoring, with a PPI variant for interaction pair design.

License: Apache-2.0 · Molecules: protein · Tasks: sequence_generation, embedding, property_prediction, sequence_optimization

Actions: generate, encode, score · Variants: 3

At a glance¶

Use it when

You need to design specific protein regions (loops, linkers, binding sites) while keeping the rest of the sequence fixed — DSM's masked infilling is uniquely suited for scaffold-constrained design
You need protein generation, embedding extraction, and sequence scoring from a single model deployment without running multiple separate models
You need to generate interacting protein pairs for biosensor design, synthetic signaling circuits, or protein-protein interaction engineering (PPI variant)
You want bidirectional context during sequence generation, where the model considers both preceding and following residues during design
You need unconditional protein sequence generation to explore protein sequence space broadly without seed constraints
You are comparing diffusion-based and autoregressive protein generation approaches and need a discrete diffusion baseline
You need configurable generation strategies (remasking) to balance quality and diversity for experimental library design

Strengths

Three-action API (generate + encode + score) provides sequence generation, embedding extraction, and fitness scoring in a single model deployment — the most versatile protein generation model in this catalog
Masked diffusion enables three generation modes — unconditional, masked infilling (fix scaffold regions while designing loops/linkers), and conditional prefix extension — offering design flexibility that autoregressive models cannot match
PPI-specialized variant (650M-PPI) trained on STRING protein-protein interaction data enables generation of interacting protein pairs, a unique capability in this catalog
Two size variants (150M, 650M) with base and PPI sub-variants provide deployment flexibility for different compute budgets and use cases
Diffusion-based generation can produce higher-quality sequences than autoregressive models for certain tasks by considering bidirectional context during the denoising process
Configurable remasking strategies (random, low-confidence, low-logit, dual) allow fine-grained control over the generation process, enabling quality-diversity tradeoffs
Apache-2.0 license permits unrestricted commercial and academic use
Recent model (2025) incorporating the latest advances in discrete diffusion modeling for proteins

Limitations

Very recent model (2025) with no independent benchmarking or applied literature — real-world validation and reliability are unproven
Higher compute requirements than other protein generators — requires A10G GPU even for the 150M variant, compared to CPU-only (MPNN) or T4 (ProGen2, ZymCTRL) alternatives
No function conditioning — cannot specify target enzymatic activity (EC number) or other functional properties during generation, unlike ZymCTRL
No structure conditioning — generates sequences from sequence context only, without 3D backbone geometry input; cannot guarantee structural compatibility with a target fold
Diffusion sampling is iterative and slower than single-pass autoregressive generation for producing individual sequences
PPI variant only available at 650M size; the 3B variant and PPI-3B are not yet released on HuggingFace (as of March 2026; check HuggingFace for current availability)
Generated sequences require extensive downstream validation — no guarantee of function, folding, or stability
Less established community and tooling ecosystem compared to ESM-2 for embeddings or ProGen2 for generation

Reach for something else when

You need a well-validated model with extensive independent benchmarks and published downstream applications (use esm2 for embeddings or progen2 for generation)
You need enzyme generation conditioned on a specific EC number (use zymctrl, which provides direct EC-number conditioning)
You need structure-conditioned sequence design from a target backbone (use mpnn for comprehensive inverse folding or esm_if1 for lightweight inverse folding)
You need the lowest-cost protein generation (use mpnn on CPU or progen2 on T4, both cheaper than DSM on A10G)
You need antibody-specific sequence generation or design (use progen2 OAS variant or antifold)
You need the broadest protein embedding ecosystem with downstream tools (use esm2, which has ESMFold and a broad ecosystem of downstream models)
You need fast single-sequence generation where autoregressive models (ProGen2) produce results faster than iterative diffusion
You need a model with a proven experimental validation track record for generated sequences (use progen2 or zymctrl, both published with experimental evidence)

Alternatives

Model	Better when	Worse when
progen2	Established publication (Cell Systems 2023), four specialized variants including antibody (OAS), bidirectional log-likelihood scoring, and extensive benchmarking.	Autoregressive-only — no masked infilling or bidirectional generation; no embedding endpoint; no PPI variant; less flexible generation modes.
zymctrl	EC-number conditioned enzyme generation with experimental validation of catalytic activity; enzyme-specialized training on 37M sequences.	Enzyme-only, generation conditioned on EC number rather than free protein-sequence context; no scoring endpoint; no masked infilling; no PPI variant.
esm2	Most established protein embeddings with five size variants, widest downstream tool ecosystem, and extensive benchmarking; overlaps dsm's encode + score on identical protein-sequence input.	Cannot generate protein sequences; encoder-only architecture limited to embedding and scoring; no PPI-specific training.

API & schema¶

Variants

Variant	Endpoint slug	GPU	CPU	Memory
150m-base	`dsm-150m-base`	a10g	4.0	16 GB
650m-base	`dsm-650m-base`	a10g	8.0	32 GB
650m-ppi	`dsm-650m-ppi`	a10g	8.0	32 GB

Call an action with POST /api/v1/{slug}/{action} — the request envelope is {"items": [...], "params": {...}} and a success returns {"results": [...]}. See the HTTP API page for the base URL, error shape, and full contract.

`generate`¶

Call it

curl -X POST http://127.0.0.1:8000/api/v1/dsm-150m-base/generate \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {}
  ]
}'

Request — DSMGenerateRequest

Field	Type	Required	Constraints	Description
`params`	DSMGenerateRequestParams	no		Optional parameters controlling this action (defaults are used when omitted).
`items`	list[DSMGenerateRequestItem]	yes	items 1–1	Batch of inputs to process in a single request. Up to 1 sequence per request.

Nested types

DSMGenerateRequestItem

Field	Type	Required	Constraints	Description
`sequence`	string	no	len –2048; default ``	Input sequence. Three modes: (1) empty string — unconditional generation; the model creates a canvas of `max_length` mask tokens (default 100) and denoises them; (2) sequence containing `<mask>` tokens — masked infilling; the model fills only the masked positions; output length equals the number of tokens in the input; (3) plain amino-acid prefix — conditional generation from the prefix; the model denoises any remaining context.

DSMGenerateRequestParams

Field	Type	Required	Constraints	Description
`num_sequences`	integer	no	≥1; ≤32; default `1`	Number of sequences to generate per input.
`temperature`	number	no	≥0.1; ≤2.0; default `1.0`	Sampling temperature; higher values increase diversity.
`max_length`	integer \| null	no		Canvas size (number of mask tokens) for unconditional generation when sequence is empty. Defaults to 100 if not specified. Ignored for masked infilling and conditional modes, where generation length is determined by the number of tokens in the input.
`step_divisor`	integer	no	≥1; ≤1000; default `100`	Diffusion step divisor; lower values yield more denoising steps and better quality at higher compute cost.
`remasking`	DSMRemaskingStrategy	no	default `random`	Remasking strategy controlling which positions are re-masked between diffusion steps.
`seed`	integer \| null	no		Random seed for reproducible sampling.

DSMRemaskingStrategy

Allowed values: low_confidence, random, low_logit, dual

Raw JSON Schema

{
  "$defs": {
    "DSMGenerateRequestItem": {
      "additionalProperties": false,
      "description": "Single input for DSM generation - can be masked or empty for unconditional.",
      "properties": {
        "sequence": {
          "default": "",
          "description": "Input sequence. Three modes: (1) empty string \u2014 unconditional generation; the model creates a canvas of `max_length` mask tokens (default 100) and denoises them; (2) sequence containing `<mask>` tokens \u2014 masked infilling; the model fills only the masked positions; output length equals the number of tokens in the input; (3) plain amino-acid prefix \u2014 conditional generation from the prefix; the model denoises any remaining context.",
          "maxLength": 2048,
          "title": "Sequence",
          "type": "string"
        }
      },
      "title": "DSMGenerateRequestItem",
      "type": "object"
    },
    "DSMGenerateRequestParams": {
      "additionalProperties": false,
      "description": "Parameters for DSM generation.",
      "properties": {
        "num_sequences": {
          "default": 1,
          "description": "Number of sequences to generate per input.",
          "maximum": 32,
          "minimum": 1,
          "title": "Num Sequences",
          "type": "integer"
        },
        "temperature": {
          "default": 1.0,
          "description": "Sampling temperature; higher values increase diversity.",
          "maximum": 2.0,
          "minimum": 0.1,
          "title": "Temperature",
          "type": "number"
        },
        "max_length": {
          "anyOf": [
            {
              "maximum": 2048,
              "minimum": 10,
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Canvas size (number of mask tokens) for unconditional generation when sequence is empty. Defaults to 100 if not specified. Ignored for masked infilling and conditional modes, where generation length is determined by the number of <mask> tokens in the input.",
          "title": "Max Length"
        },
        "step_divisor": {
          "default": 100,
          "description": "Diffusion step divisor; lower values yield more denoising steps and better quality at higher compute cost.",
          "maximum": 1000,
          "minimum": 1,
          "title": "Step Divisor",
          "type": "integer"
        },
        "remasking": {
          "$ref": "#/$defs/DSMRemaskingStrategy",
          "default": "random",
          "description": "Remasking strategy controlling which positions are re-masked between diffusion steps."
        },
        "seed": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Random seed for reproducible sampling.",
          "title": "Seed"
        }
      },
      "title": "DSMGenerateRequestParams",
      "type": "object"
    },
    "DSMRemaskingStrategy": {
      "description": "Remasking strategy for DSM generation.",
      "enum": [
        "low_confidence",
        "random",
        "low_logit",
        "dual"
      ],
      "title": "DSMRemaskingStrategy",
      "type": "string"
    }
  },
  "additionalProperties": false,
  "description": "Request for DSM sequence generation.",
  "properties": {
    "params": {
      "$ref": "#/$defs/DSMGenerateRequestParams",
      "description": "Optional parameters controlling this action (defaults are used when omitted)."
    },
    "items": {
      "description": "Batch of inputs to process in a single request. Up to 1 sequence per request.",
      "items": {
        "$ref": "#/$defs/DSMGenerateRequestItem"
      },
      "maxItems": 1,
      "minItems": 1,
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "DSMGenerateRequest",
  "type": "object"
}

Response — DSMGenerateResponse

Field	Type	Required	Constraints	Description
`results`	list[list[DSMGenerateResponseResult]]	yes		Per-input results in request order. Each inner list contains num_sequences generated sequences.

Nested types

DSMGenerateResponseResult

Field	Type	Required	Description
`sequence`	string	yes	Generated protein sequence in single-letter amino-acid codes.
`log_prob`	number	yes	Pseudo-log-likelihood of the sequence under the model.
`perplexity`	number	yes	Perplexity of the sequence under the model (lower means more likely).
`sequence2`	string \| null	no	Second generated sequence for PPI-variant outputs; None for base-model generations.

Raw JSON Schema

{
  "$defs": {
    "DSMGenerateResponseResult": {
      "description": "Result for a single generated sequence.",
      "properties": {
        "sequence": {
          "description": "Generated protein sequence in single-letter amino-acid codes.",
          "title": "Sequence",
          "type": "string"
        },
        "log_prob": {
          "description": "Pseudo-log-likelihood of the sequence under the model.",
          "title": "Log Prob",
          "type": "number"
        },
        "perplexity": {
          "description": "Perplexity of the sequence under the model (lower means more likely).",
          "title": "Perplexity",
          "type": "number"
        },
        "sequence2": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Second generated sequence for PPI-variant outputs; None for base-model generations.",
          "title": "Sequence2"
        }
      },
      "required": [
        "sequence",
        "log_prob",
        "perplexity"
      ],
      "title": "DSMGenerateResponseResult",
      "type": "object"
    }
  },
  "properties": {
    "results": {
      "description": "Per-input results in request order. Each inner list contains num_sequences generated sequences.",
      "items": {
        "items": {
          "$ref": "#/$defs/DSMGenerateResponseResult"
        },
        "type": "array"
      },
      "title": "Results",
      "type": "array"
    }
  },
  "required": [
    "results"
  ],
  "title": "DSMGenerateResponse",
  "type": "object"
}

`encode`¶

Call it

curl -X POST http://127.0.0.1:8000/api/v1/dsm-150m-base/encode \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {
      "sequence": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE"
    }
  ]
}'

Request — DSMEncodeRequest

Field	Type	Required	Constraints	Description
`params`	DSMEncodeRequestParams	no		Optional parameters controlling this action (defaults are used when omitted).
`items`	list[DSMEncodeRequestItem]	yes	items 1–16	Batch of inputs to process in a single request. Up to 16 sequences per request.

Nested types

DSMEncodeIncludeOptions

Allowed values: mean, per_residue, cls

DSMEncodeRequestItem

Field	Type	Required	Constraints	Description
`sequence`	string	yes	len 1–2048	A protein sequence in single-letter amino-acid codes.

DSMEncodeRequestParams

Field	Type	Required	Constraints	Description
`include`	list[DSMEncodeIncludeOptions]	no	default `['mean']`	Optional outputs to compute and include in the response.

Raw JSON Schema

{
  "$defs": {
    "DSMEncodeIncludeOptions": {
      "enum": [
        "mean",
        "per_residue",
        "cls"
      ],
      "title": "DSMEncodeIncludeOptions",
      "type": "string"
    },
    "DSMEncodeRequestItem": {
      "additionalProperties": false,
      "properties": {
        "sequence": {
          "description": "A protein sequence in single-letter amino-acid codes.",
          "maxLength": 2048,
          "minLength": 1,
          "title": "Sequence",
          "type": "string"
        }
      },
      "required": [
        "sequence"
      ],
      "title": "DSMEncodeRequestItem",
      "type": "object"
    },
    "DSMEncodeRequestParams": {
      "additionalProperties": false,
      "properties": {
        "include": {
          "description": "Optional outputs to compute and include in the response.",
          "items": {
            "$ref": "#/$defs/DSMEncodeIncludeOptions"
          },
          "title": "Include",
          "type": "array",
          "default": [
            "mean"
          ]
        }
      },
      "title": "DSMEncodeRequestParams",
      "type": "object"
    }
  },
  "additionalProperties": false,
  "properties": {
    "params": {
      "$ref": "#/$defs/DSMEncodeRequestParams",
      "description": "Optional parameters controlling this action (defaults are used when omitted)."
    },
    "items": {
      "description": "Batch of inputs to process in a single request. Up to 16 sequences per request.",
      "items": {
        "$ref": "#/$defs/DSMEncodeRequestItem"
      },
      "maxItems": 16,
      "minItems": 1,
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "DSMEncodeRequest",
  "type": "object"
}

Response — DSMEncodeResponse

Field	Type	Required	Constraints	Description
`results`	list[DSMEncodeResponseResult]	yes		Per-input results, returned in the same order as the request items.

Nested types

DSMEncodeResponseResult

Field	Type	Required	Description
`sequence_index`	integer	yes	Index of the corresponding input sequence within the request batch.
`embeddings`	list[number] \| null	no	Mean-pooled embedding vector for the sequence.
`residue_embeddings`	list[list[number]] \| null	no	Per-residue embedding vectors.
`cls_embeddings`	list[number] \| null	no	CLS-token embedding vector for the sequence.

Raw JSON Schema

{
  "$defs": {
    "DSMEncodeResponseResult": {
      "properties": {
        "sequence_index": {
          "description": "Index of the corresponding input sequence within the request batch.",
          "title": "Sequence Index",
          "type": "integer"
        },
        "embeddings": {
          "anyOf": [
            {
              "items": {
                "type": "number"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Mean-pooled embedding vector for the sequence.",
          "title": "Embeddings"
        },
        "residue_embeddings": {
          "anyOf": [
            {
              "items": {
                "items": {
                  "type": "number"
                },
                "type": "array"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Per-residue embedding vectors.",
          "title": "Residue Embeddings"
        },
        "cls_embeddings": {
          "anyOf": [
            {
              "items": {
                "type": "number"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "CLS-token embedding vector for the sequence.",
          "title": "Cls Embeddings"
        }
      },
      "required": [
        "sequence_index"
      ],
      "title": "DSMEncodeResponseResult",
      "type": "object"
    }
  },
  "properties": {
    "results": {
      "description": "Per-input results, returned in the same order as the request items.",
      "items": {
        "$ref": "#/$defs/DSMEncodeResponseResult"
      },
      "title": "Results",
      "type": "array"
    }
  },
  "required": [
    "results"
  ],
  "title": "DSMEncodeResponse",
  "type": "object"
}

`score`¶

Call it

curl -X POST http://127.0.0.1:8000/api/v1/dsm-150m-base/score \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {
      "sequence": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE"
    }
  ]
}'

Request — DSMScoreRequest

Field	Type	Required	Constraints	Description
`items`	list[DSMScoreRequestItem]	yes	items 1–16	Batch of inputs to process in a single request. Up to 16 sequences per request.

Nested types

DSMScoreRequestItem

Field	Type	Required	Constraints	Description
`sequence`	string	yes	len 1–2048	A protein sequence in single-letter amino-acid codes.

Raw JSON Schema

{
  "$defs": {
    "DSMScoreRequestItem": {
      "additionalProperties": false,
      "properties": {
        "sequence": {
          "description": "A protein sequence in single-letter amino-acid codes.",
          "maxLength": 2048,
          "minLength": 1,
          "title": "Sequence",
          "type": "string"
        }
      },
      "required": [
        "sequence"
      ],
      "title": "DSMScoreRequestItem",
      "type": "object"
    }
  },
  "additionalProperties": false,
  "properties": {
    "items": {
      "description": "Batch of inputs to process in a single request. Up to 16 sequences per request.",
      "items": {
        "$ref": "#/$defs/DSMScoreRequestItem"
      },
      "maxItems": 16,
      "minItems": 1,
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "DSMScoreRequest",
  "type": "object"
}

Response — DSMScoreResponse

Field	Type	Required	Constraints	Description
`results`	list[DSMScoreResponseResult]	yes		Per-input results, returned in the same order as the request items.

Nested types

DSMScoreResponseResult

Field	Type	Required	Description
`log_prob`	number	yes	Pseudo-log-likelihood of the sequence under the model.
`perplexity`	number	yes	Perplexity of the sequence under the model (lower means more likely).
`sequence_length`	integer	yes	Length of the input sequence in amino acids.

Raw JSON Schema

{
  "$defs": {
    "DSMScoreResponseResult": {
      "properties": {
        "log_prob": {
          "description": "Pseudo-log-likelihood of the sequence under the model.",
          "title": "Log Prob",
          "type": "number"
        },
        "perplexity": {
          "description": "Perplexity of the sequence under the model (lower means more likely).",
          "title": "Perplexity",
          "type": "number"
        },
        "sequence_length": {
          "description": "Length of the input sequence in amino acids.",
          "title": "Sequence Length",
          "type": "integer"
        }
      },
      "required": [
        "log_prob",
        "perplexity",
        "sequence_length"
      ],
      "title": "DSMScoreResponseResult",
      "type": "object"
    }
  },
  "properties": {
    "results": {
      "description": "Per-input results, returned in the same order as the request items.",
      "items": {
        "$ref": "#/$defs/DSMScoreResponseResult"
      },
      "title": "Results",
      "type": "array"
    }
  },
  "required": [
    "results"
  ],
  "title": "DSMScoreResponse",
  "type": "object"
}

Usage¶

One-line summary: Masked diffusion protein language model for sequence generation (unconditional, masked infilling, conditional), embedding extraction, and log-probability scoring, with a PPI variant for interaction pair design.

Overview¶

DSM is a novel Protein Language Model (pLM) developed by Gleghorn Lab and Synthyra. It was trained with masked diffusion to enable both high-quality representation learning and generative protein design.

Capabilities¶

Generate (generate): Generate protein sequences via masked diffusion - supports unconditional generation (empty input), masked sequence filling (<mask> tokens), and conditional generation from a prefix
Encode (encode): Extract embeddings (mean-pooled, per-residue, or CLS token) for similarity search, clustering, or downstream ML tasks
Score (score): Calculate log probabilities and perplexity for sequence quality assessment, filtering, and confidence scoring

Architecture¶

Property	Value
Architecture	Transformer + masked diffusion
Training objective	Masked diffusion (iterative denoising)
Training data	omg_prot50 (base), STRING (PPI)
Max sequence length	2048 tokens

Capabilities & Limitations¶

CAN be used for: - Unconditional protein sequence generation - Masked sequence infilling (fixing regions, generating others) - Conditional sequence generation from prefix - Protein-protein interaction pair design (PPI variant) - Embedding extraction (mean-pooled, per-residue, CLS) - Sequence quality scoring (log probability, perplexity)

CANNOT be used for: - Structure-conditioned generation (sequence-only) - Non-protein molecules - Multi-chain complex design (except PPI dual sequences)

Usage Examples¶

Generate Sequences (Unconditional)¶

Pass an empty string and set max_length to control the number of residues generated. The model creates a canvas of that many <mask> tokens and denoises them.

from models.dsm.schema import (
    DSMGenerateRequest,
    DSMGenerateRequestItem,
    DSMGenerateRequestParams,
    DSMRemaskingStrategy,
)

request = DSMGenerateRequest(
    params=DSMGenerateRequestParams(
        num_sequences=5,
        temperature=1.0,
        max_length=100,  # Generate sequences of ~100 residues
        remasking=DSMRemaskingStrategy.RANDOM,
        seed=42,
    ),
    items=[
        DSMGenerateRequestItem(sequence=""),  # Empty triggers unconditional generation
    ],
)

Generate Sequences (Masked Infilling)¶

from models.dsm.schema import (
    DSMGenerateRequest,
    DSMGenerateRequestItem,
    DSMGenerateRequestParams,
)

request = DSMGenerateRequest(
    params=DSMGenerateRequestParams(num_sequences=3),
    items=[
        DSMGenerateRequestItem(
            sequence="MKTL<mask><mask><mask>VLGK",  # Fill in masked positions
        ),
    ],
)

Encode Sequences¶

from models.dsm.schema import (
    DSMEncodeRequest,
    DSMEncodeRequestItem,
    DSMEncodeRequestParams,
    DSMEncodeIncludeOptions,
)

request = DSMEncodeRequest(
    params=DSMEncodeRequestParams(
        include=[DSMEncodeIncludeOptions.MEAN, DSMEncodeIncludeOptions.CLS],
    ),
    items=[
        DSMEncodeRequestItem(sequence="MKTLLLTLVVVTLVL"),
    ],
)

Score Sequences¶

from models.dsm.schema import (
    DSMScoreRequest,
    DSMScoreRequestItem,
)

request = DSMScoreRequest(
    items=[
        DSMScoreRequestItem(sequence="MKTLLLTLVVVTLVL"),
    ],
)

Architecture & training¶

Architecture¶

Model Type & Innovation¶

DSM (Diffusion Sequence Model) is a protein language model trained with masked diffusion for both generative protein design and representation learning. Unlike autoregressive models (e.g., ProGen2) that generate sequences left-to-right, DSM uses an iterative denoising process that fills masked positions over multiple steps, enabling conditional and unconditional sequence generation.

The key innovation is the masked diffusion training objective, which unifies the strengths of masked language models (bidirectional context, good embeddings) and diffusion models (high-quality generation). DSM progressively unmasks a fully masked sequence over configurable steps, with remasking strategies that control which positions are denoised at each step.

Parameters & Layers¶

Variant	Parameters	Hidden Dim	Layers	Attention Heads	GPU	Memory
DSM-150M	150M	640	30	20	A10G	16 GB
DSM-650M	650M	1280	33	20	A10G	32 GB
DSM-3B	3B	2560	36	40	A100	64 GB (not yet released)

DSM models are extended from pre-trained ESM2 checkpoints (Hallee et al., 2025). DSM-150M inherits the ESM2-150M architecture (30 layers, 640 hidden dim, 20 attention heads). DSM-650M inherits the ESM2-650M architecture (33 layers, 1280 hidden dim, 20 attention heads). Both were trained on OMGprot50, a dataset of over 207 million protein sequences clustered at 50% sequence identity from the Open MetaGenomic dataset (OMG). DSM-150M was trained for 100,000 steps with batch size 32 and max sequence length 512. DSM-650M was trained for 100,000 steps with batch size 128 and max sequence length 2048. DSMppi (the PPI variant) was fine-tuned from DSM-650M on protein-protein interaction pairs from the STRING database.

All variants share:

Property	Value
Max sequence length	2048 tokens
Vocabulary	Standard amino acid tokenizer
Base architecture	Transformer with ESM backbone

Training Data¶

Variant	Dataset	Description
Base (150M, 650M)	omg_prot50	General protein sequences
PPI (650M)	STRING database	Protein-protein interaction pairs

Loss Function & Objective¶

Masked diffusion objective: the model learns to predict the original amino acid at masked positions through iterative denoising. During training, a random fraction of positions are masked and the model predicts the original tokens. The diffusion schedule controls the masking fraction over time.

Tokenization / Input Processing¶

Property	Details
Tokenizer	ESM-style character-level tokenizer
Special tokens	`<mask>`, `<eos>`, standard BOS/EOS
Mask token	`<mask>` for infilling positions
Separator	`<eos>` between sequences (PPI variant)
Max length	2048 tokens

Performance & Benchmarks¶

Published Benchmarks¶

From Hallee et al. (2025): DSM models match or outperform MLM-based and discrete diffusion pLMs (DPLM) of the same size, as well as an autoregressive pLM (ProtCLM-1B) almost twice DSM's size, on downstream representation tasks. DSM was benchmarked against ESM2, GLM2, ProtBert, ProtCLM-1B, ESMC, DPLM, ANKH, and ProtT5 using linear probes on supervised datasets with mean-pooled last hidden state embeddings. On generation quality, DSM produces biomimetic sequences with amino acid compositions, predicted secondary structures, and predicted functions that closely match natural protein distributions, even at 90% token corruption. DSMppi produces protein binder candidates with superior predicted binding affinity compared to known binders on the Bench-tested Binder Benchmark (BenchBB).

BioLM Verification Results¶

Test Case	Tolerance	Status
Unconditional generation	Structure validation	PASS
Masked generation	Structure validation	PASS
Conditional generation	Structure validation	PASS
Mean-pooled embeddings	rel_tol 1e-4, cosine < 0.02	PASS
Per-residue embeddings	rel_tol 1e-4, cosine < 0.02	PASS
Sequence scoring	rel_tol 1e-4	PASS

Generation tests validate output structure (valid amino acids, reasonable perplexity) rather than exact sequence matching, since generation is stochastic by design.

Comparison to Alternatives¶

Model	Type	Key Advantage	Key Disadvantage
DSM (this)	Masked diffusion	Unified generation + embedding model	Newer, fewer benchmarks
ProGen2	Autoregressive	Established generation quality	Left-to-right only, no bidirectional embeddings
ESM2	Masked LM	Best single-sequence embeddings	No generation capability
EvoDiff	Discrete diffusion	Structure-conditioned generation	Requires structural guidance
ESM3	Multimodal	Structure + function + sequence	More complex, larger resource requirements

Error Bars & Confidence¶

Generation is inherently stochastic. When seed is provided in the request, generation is reproducible. When seed is None, time-based entropy is used for diversity. Embedding and scoring are deterministic.

Strengths & Limitations¶

Pros¶

Unified model for both generation and representation learning
Three generation modes: unconditional, masked infilling, conditional from prefix
PPI variant for protein-protein interaction design
Configurable remasking strategies (random, low_confidence, low_logit, dual)
Multiple size variants (150M to 3B) for speed/quality tradeoff
Embeddings competitive with dedicated embedding models

Cons¶

Generate batch size limited to 1 item per request (diffusion is compute-intensive)
No structure conditioning (sequence-only generation)
PPI variant only available for 650M size
3B variant not yet released on HuggingFace
Generation quality depends on step_divisor parameter tuning

Known Failure Modes¶

Very long generation: Generating sequences close to 2048 tokens may produce lower quality output
PPI dual decode failures: PPI variant's decode_dual_input may fail for some inputs; falls back to single-sequence decode
Empty input validation: Empty sequences are allowed for unconditional generation; non-empty sequences without <mask> are treated as conditional

Implementation Details¶

Inference Pipeline¶

Generate pipeline:

Request
  |-- 1. Set random seed (user-provided or time-based)
  |-- 2. Tokenize input sequence
  |-- 3. Repeat input for num_sequences
  |-- 4. Run mask_diffusion_generate with remasking strategy
  |-- 5. Decode output sequences (dual decode for PPI)
  |-- 6. Calculate log_prob and perplexity for each
  |-- 7. Return DSMGenerateResponse

Encode pipeline:

Request
  |-- 1. Tokenize sequences with special tokens
  |-- 2. Forward pass through ESM backbone
  |-- 3. Extract hidden states
  |-- 4. Post-process: mean pooling, per-residue, or CLS token
  |-- 5. Return DSMEncodeResponse

Score pipeline:

Request
  |-- 1. Tokenize sequence
  |-- 2. Forward pass -> logits
  |-- 3. Compute log-softmax
  |-- 4. Sum position-aligned pseudo-log-probabilities (bidirectional, non-AR)
  |-- 5. Calculate perplexity
  |-- 6. Return DSMScoreResponse

Memory & Compute Profile¶

Variant	GPU	VRAM	Generate Latency	Encode Latency
DSM-150M	A10G	~4 GB	~500ms/seq	~100ms/seq
DSM-650M	A10G	~12 GB	~1-2s/seq	~150ms/seq
DSM-3B	A100	~24 GB	~5-10s/seq (not yet released; illustrative)	~500ms/seq

Determinism & Reproducibility¶

Setting	Value
`torch.manual_seed`	User seed or time-based
`random.seed`	Same as torch seed
`numpy.random.seed`	Same as torch seed
`torch.cuda.manual_seed_all`	Same as torch seed
Generation	Deterministic when seed is provided
Embedding/scoring	Always deterministic

Caching Behavior¶

Response caching is handled externally (e.g., by a caching proxy), not inside the model container. Note: generation with time-based seed will not benefit from caching since inputs differ.

Versions & Changelog¶

Version	Date	Changes
v1	2025-12-06	Initial implementation with generate, encode, and score actions
v1 (updated)	2025-12-11	Added PPI variant support with dual-sequence decode
v1 (updated)	2026-01-16	Pinned DSM repo to commit `ca7b5c8c` for reproducibility

Biology¶

Molecule Coverage¶

Primary Molecule Type(s)¶

DSM is trained on protein sequences from diverse organisms. The base variants are trained on the omg_prot50 dataset (general protein sequences), while the PPI variant is trained on STRING protein-protein interaction data.

Performance characteristics by protein type:

Globular proteins: Primary training domain. Best generation quality and embedding accuracy.
Enzymes: Well-represented in training data. Generated sequences likely to be catalytically plausible.
Protein-protein interactors: The PPI variant (650M) is specifically trained on STRING interaction pairs, generating sequences designed to interact with partner proteins.
Membrane proteins: Present in training data but may be under-represented.
Intrinsically disordered proteins: The model can encode and score these, but generation quality may be lower since disordered sequences have less structural constraint.
Peptides: Short sequences (< 30 residues) provide limited context for the diffusion process.

Cross-Applicability¶

Molecule Type	Applicability	Evidence	Caveats
Globular proteins	High	Primary training domain (omg_prot50)	Best performance on canonical folds
Enzymes	High	Well-represented in training data	No explicit catalytic mechanism modeling
Protein interaction pairs	High (PPI variant)	Trained on STRING database	PPI variant only for 650M
Antibodies	Moderate	Protein sequences in training data	No CDR-specific training
Membrane proteins	Moderate	Present in training data	May be under-represented
Peptides	Low	Short sequences lack context	Consider peptide-specific models
DNA/RNA	Not supported	Not in training data	Protein sequences only

Biological Problems Addressed¶

Protein Sequence Design¶

Problem: Designing novel protein sequences with desired properties is a fundamental challenge in biotechnology. Traditional directed evolution is slow and expensive; computational sequence generation can accelerate the design cycle.

How DSM helps: The generate action produces novel protein sequences through masked diffusion. Three modes are available: - Unconditional: Generate entirely new sequences (exploring protein sequence space) - Masked infilling: Fix known regions and generate the unknown (e.g., design a loop while keeping the scaffold) - Conditional: Extend a sequence prefix (e.g., generate the rest of a protein given the N-terminal domain)

Biological meaning: Generated sequences are drawn from the learned distribution of natural proteins. They are expected to encode foldable proteins with reasonable physicochemical properties. The log probability and perplexity scores indicate how "natural" each generated sequence appears to the model.

Protein-Protein Interaction Design¶

Problem: Engineering proteins that specifically interact with target partners is essential for therapeutic antibodies, biosensors, and synthetic biology circuits.

How DSM helps: The PPI variant (DSM-650M-PPI, trained on STRING) generates protein pairs designed to interact. It uses a dual-sequence format with <eos> separator, outputting both an interactor sequence and its partner.

Biological meaning: Sequences generated by the PPI variant have been trained to recapitulate interaction patterns from the STRING database -- they encode surface complementarity, electrostatic matching, and hydrophobic interface features typical of natural protein-protein interactions.

Protein Representation Learning¶

Problem: Many downstream protein analysis tasks require numerical representations. Embeddings should capture evolutionary, structural, and functional information from sequence alone.

How DSM helps: The encode action extracts dense vector embeddings (mean-pooled, per-residue, or CLS token) from DSM's internal representations. These can be used as features for downstream classifiers.

Biological meaning: DSM embeddings capture evolutionary and biophysical patterns learned during masked diffusion training. Proteins with similar function or structure will have similar embeddings, even at low sequence identity.

Sequence Quality Assessment¶

Problem: Evaluating whether a protein sequence (natural or designed) is likely to encode a functional, foldable protein.

How DSM helps: The score action computes the log probability and perplexity of a sequence. Lower perplexity indicates the sequence is more consistent with patterns learned from natural proteins.

Biological meaning: Perplexity serves as a proxy for sequence "naturalness." Sequences with high perplexity contain unusual amino acid patterns that may indicate design errors, frameshifts, or evolutionary divergence. This can be used to filter generated sequences or rank natural variants.

Applied Use Cases¶

DSM is a recent model (2025). Applied use cases include:

De novo protein design: Generating novel protein sequences for functional screening
Masked region design: Designing specific regions (loops, linkers, binding sites) while preserving scaffold
Interaction pair generation: Using the PPI variant to generate interacting protein pairs
Embedding extraction: Using DSM embeddings as features for downstream prediction tasks
Sequence scoring: Ranking candidate sequences by naturalness

Anticipated (not yet published) use cases:

Combinatorial library design guided by DSM scoring
Multi-round design with iterative generation and experimental feedback
Transfer learning from DSM embeddings for property prediction

Complementary Models¶

ESM2 (this catalog): DSM uses an ESM-style backbone. ESM2 provides established embeddings for comparison. DSM offers generation capability that ESM2 lacks.
SPURS (this catalog): Can score DSM-generated sequences for stability impact of specific mutations.
ESMFold / Chai-1 (this catalog): Can predict 3D structures of DSM-generated sequences to validate foldability.

Alternative Models¶

Alternative	Advantage Over DSM	Disadvantage vs DSM
ProGen2	Established autoregressive generation	No bidirectional embeddings
ESM2	Best single-sequence embeddings	No generation capability
ESM3	Multimodal (sequence + structure + function)	Larger resource requirements
EvoDiff	Structure-conditioned generation	Requires structural input
ProtGPT2	Simple autoregressive generation	Lower quality than diffusion-based

When to choose DSM: Use DSM when you need both generation and embedding in a single model, or when you need masked infilling / conditional generation rather than left-to-right autoregressive.

When to choose alternatives: Use ESM2 for embeddings only (more established); use ProGen2 for autoregressive generation; use ESM3 for structure-conditioned design.

Biological Background¶

Protein sequence design is the computational challenge of generating amino acid sequences that fold into desired 3D structures and perform target functions. This is the inverse of the protein folding problem: rather than predicting structure from sequence, design predicts sequence from desired properties.

Masked diffusion for proteins: DSM adapts the diffusion modeling paradigm (successful in image generation) to discrete protein sequences. Starting from a fully masked sequence, the model iteratively unmasks positions, predicting the most likely amino acid at each step based on the current (partially revealed) context. This process is analogous to an artist sketching a painting -- first laying down broad strokes, then refining details.

Remasking strategies control which positions are denoised at each step: - Random: Uniformly random position selection (default) - Low confidence: Preferentially unmask positions where the model is most confident - Low logit: Unmask positions with lowest predicted probability - Dual: Combines confidence and logit criteria

Key terminology: - Diffusion model: A generative model that learns to reverse a corruption process (masking, noise addition) - Unconditional generation: Generating sequences without any input constraint - Masked infilling: Generating specific positions while fixing others (like fill-in-the-blank) - Conditional generation: Generating sequence extensions from a given prefix - Perplexity: Exponential of average negative log-probability; measures how "surprised" the model is by a sequence. Lower = more natural. - Log probability: Sum of log P(residue_i | context) across all positions; more negative = less likely - STRING database: A database of known and predicted protein-protein interactions from which the PPI variant learns

Sources & license¶

License: Apache-2.0 (text)

Papers

Diffusion Sequence Models for Enhanced Protein Representation and Generation — arXiv preprint, 2025 · arXiv

Source repositories

github: https://github.com/GleghornLab/DSM
huggingface: https://huggingface.co/GleghornLab/DSM_650

Cite

@article{hallee2025dsm,
  title={Diffusion Sequence Models for Enhanced Protein Representation and Generation},
  author={Hallee, Logan and Rafailidis, Nikolaos and Bichara, David B. and Gleghorn, Jason P.},
  journal={arXiv preprint},
  eprint={2506.08293},
  year={2025}
}

DSM¶

At a glance¶

API & schema¶

generate¶

encode¶

score¶

Usage¶

Overview¶

Capabilities¶

Architecture¶

Capabilities & Limitations¶

Usage Examples¶

Generate Sequences (Unconditional)¶

Generate Sequences (Masked Infilling)¶

Encode Sequences¶

Score Sequences¶

Architecture & training¶

Architecture¶

Model Type & Innovation¶

Parameters & Layers¶

Training Data¶

Loss Function & Objective¶

Tokenization / Input Processing¶

Performance & Benchmarks¶

Published Benchmarks¶

BioLM Verification Results¶

Comparison to Alternatives¶

Error Bars & Confidence¶

Strengths & Limitations¶

Pros¶

Cons¶

Known Failure Modes¶

Implementation Details¶

Inference Pipeline¶

Memory & Compute Profile¶

Determinism & Reproducibility¶

Caching Behavior¶

Versions & Changelog¶

Biology¶

Molecule Coverage¶

Primary Molecule Type(s)¶

Cross-Applicability¶

Biological Problems Addressed¶

Protein Sequence Design¶

Protein-Protein Interaction Design¶

Protein Representation Learning¶

Sequence Quality Assessment¶

Applied Use Cases¶

Related Models¶

Complementary Models¶

Alternative Models¶

Biological Background¶

Sources & license¶

`generate`¶

`encode`¶

`score`¶