E1¶

Retrieval-augmented protein encoder language model that produces embeddings, masked predictions, and log-probability scores, optionally conditioned on homologous context sequences.

License: Apache-2.0 · Molecules: protein · Tasks: embedding, sequence_completion, property_prediction

Actions: encode, predict, log_prob · Variants: 3

At a glance¶

Use it when

You have homologous sequences available and want to improve embedding quality, fitness predictions, or masked predictions beyond what single-sequence models (ESM2) can achieve
You need zero-shot variant effect scoring with evolutionary context -- provide the query protein and its homologs to log_prob for improved mutation impact assessment
You are building downstream ML models (function prediction, localization, stability estimation) and want context-enriched embeddings as features
You need masked residue prediction at specific positions (active sites, binding interfaces) and have related sequences that provide evolutionary constraints
You want to analyze protein families by encoding all members with shared context sequences for consistent, evolutionarily informed representations
You need a lightweight embedding model (T4/L4 GPU) that provides MSA-like evolutionary context without the computational cost of running JackHMMER or HHblits

Strengths

Retrieval-augmented protein representations: accepts up to 50 context sequences (homologs) alongside the query, capturing MSA-like evolutionary information via block-causal attention without requiring formal sequence alignment
Three size variants (150M, 300M, 600M) spanning T4 to L4 GPUs, providing fine-grained cost-performance tradeoff for different deployment scales and budget constraints
Block-causal attention architecture allows the query to attend to all context sequences while context sequences only attend to themselves -- efficiently simulating MSA column reading without O(N^2) alignment cost
Three complementary actions: encode (embeddings), predict (masked residue prediction), and log_prob (sequence scoring) -- covering embedding, design, and fitness prediction use cases in a single model
Zero-shot fitness prediction with context sequences improves variant effect scoring over single-sequence models, as the context provides explicit evolutionary conservation signals
Apache-2.0 license with no usage restrictions -- suitable for commercial integration and clinical variant interpretation pipelines
Extended amino acid alphabet support (20 standard + B, X, Z, U, O) handles ambiguous and non-standard residues commonly found in database entries without preprocessing
Masked prediction at specific positions enables targeted residue design: mask catalytic residues in an enzyme and use homolog context to predict evolutionarily compatible substitutions

Limitations

No structure awareness -- trained purely on protein sequences without 3D structural supervision; models like ProstT5 incorporate structure tokens and outperform on structure-dependent tasks when structures are available
Performance degrades significantly without context sequences -- when used as a single-sequence model (no homologs), E1 may not substantially outperform ESM2 on embedding tasks
Recent model (2025) with limited published downstream benchmarks and community validation compared to the extensively benchmarked ESM2 ecosystem
Encoder-only architecture -- cannot generate novel protein sequences or perform autoregressive sequence design; for generative tasks, use ProteinMPNN, ProGen2, or ZymCTRL
No structure prediction capability -- produces embeddings, logits, and log-probabilities but not 3D coordinates; use ESMFold, Chai-1, or AF2 for structure prediction
Context sequence selection requires user knowledge of relevant homologs -- the model does not perform retrieval itself; users must provide appropriate context sequences from databases like UniRef
Short peptides (< 15 residues) provide insufficient context for the transformer architecture, limiting utility for peptide-focused applications
No explicit handling of post-translational modifications (phosphorylation, glycosylation) -- modified residues are not represented in the model vocabulary

Reach for something else when

You need 3D protein structure prediction -- E1 produces embeddings and sequence-level predictions, not structures; use ESMFold, Chai-1, or AF2 NIM
You need to design or generate novel protein sequences -- E1 is an encoder-only model; use ProteinMPNN for structure-conditioned design or BoltzGen for de novo binder design
You have no homologous sequences available and want the best single-sequence embeddings -- ESM2 has broader validation and a larger ecosystem of downstream tools for single-sequence use
You need structure-aware protein representations and have access to predicted or experimental structures -- use ProstT5, which incorporates Foldseek structural tokens
You are working with non-protein molecules (DNA, RNA, small molecules) -- E1 is protein-sequence-only
You need embeddings for very short peptides (< 15 residues) where neither the query nor potential context sequences provide sufficient information
You need the widest possible ecosystem compatibility and published benchmarks -- ESM2 remains the most broadly adopted protein language model with the most downstream integrations

Alternatives

Model	Better when	Worse when
esm2	Broadest published benchmarks and downstream ecosystem, five size variants (8M-3B), 8M/35M run CPU-only.	Processes each sequence in isolation; cannot use homolog context, so E1 gives better fitness/masked predictions when context sequences are available.
esmc	Newer, more efficient single-sequence encoder with competitive embedding quality.	No retrieval-augmented (context-sequence) inference; lacks E1's block-causal MSA-like conditioning.
esm1b	Long-established 650M protein encoder with the same three actions (encode/predict/log_prob) and extensive downstream validation for embeddings and zero-shot scoring.	Single-sequence only; cannot condition on homolog context, so no evolutionary-signal boost that E1 provides.
esm1v	Purpose-built for zero-shot variant-effect prediction via masked-marginal logits, with a 5-model ensemble for robustness.	predict-only (no embeddings or log_prob action) and single-sequence; E1 adds homolog context plus encode and log_prob scoring.
prostt5	Structure-aware protein-sequence encoder (Foldseek 3Di tokens) and AA<->structure translation; stronger on structure-dependent tasks when structures are available.	No retrieval/context mechanism and no masked-prediction or log_prob scoring; E1 offers context-conditioned embeddings plus predict/log_prob.

API & schema¶

Variants

Variant	Endpoint slug	GPU	CPU	Memory
150m	`e1-150m`	t4	3.0	8 GB
300m	`e1-300m`	l4	4.0	16 GB
600m	`e1-600m`	l4	4.0	24 GB

Call an action with POST /api/v1/{slug}/{action} — the request envelope is {"items": [...], "params": {...}} and a success returns {"results": [...]}. See the HTTP API page for the base URL, error shape, and full contract.

`encode`¶

Call it

curl -X POST http://127.0.0.1:8000/api/v1/e1-150m/encode \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {
      "sequence": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE"
    }
  ]
}'

Request — E1EncodeRequest

Field	Type	Required	Constraints	Description
`params`	E1EncodeRequestParams	no		Optional parameters controlling this action (defaults are used when omitted).
`items`	list[E1EncodeRequestItem]	yes	items 1–8	Batch of inputs to process in a single request. Up to 8 sequences per request.

Nested types

E1EncodeIncludeOptions

Allowed values: mean, per_residue, per_residue, logits

E1EncodeRequestItem

Field	Type	Required	Constraints	Description
`sequence`	string	yes	len 1–2048	A protein sequence in single-letter amino-acid codes (extended alphabet: standard 20 + B/X/Z/U/O).
`context_sequences`	list[string] \| null	no		Optional homologous sequences to condition inference via block-causal attention; up to 50 sequences allowed.

E1EncodeRequestParams

Field	Type	Required	Constraints	Description
`repr_layers`	list[integer]	no	default `[-1]`	Hidden layers whose representations to return (negative indexes count from the last layer).
`include`	list[E1EncodeIncludeOptions]	no	default `['mean']`	Optional outputs to compute and include in the response.

Raw JSON Schema

{
  "$defs": {
    "E1EncodeIncludeOptions": {
      "enum": [
        "mean",
        "per_residue",
        "per_residue",
        "logits"
      ],
      "title": "E1EncodeIncludeOptions",
      "type": "string"
    },
    "E1EncodeRequestItem": {
      "additionalProperties": false,
      "description": "Single item for encoding.\n\nE1 supports retrieval-augmented inference where context sequences (homologs)\ncan be prepended to the query sequence to improve predictions.\n\nArgs:\n    sequence: The query sequence to encode (required). Accepts extended\n        amino acid alphabet (ACDEFGHIKLMNPQRSTVWY + BXZUO).\n    context_sequences: Optional list of homologous sequences for context.\n        These are prepended to the query and the model uses block-causal\n        attention to condition the query on the context. Each context\n        sequence must also be a valid amino acid sequence.",
      "properties": {
        "sequence": {
          "description": "A protein sequence in single-letter amino-acid codes (extended alphabet: standard 20 + B/X/Z/U/O).",
          "maxLength": 2048,
          "minLength": 1,
          "title": "Sequence",
          "type": "string"
        },
        "context_sequences": {
          "anyOf": [
            {
              "items": {
                "type": "string"
              },
              "maxItems": 50,
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Optional homologous sequences to condition inference via block-causal attention; up to 50 sequences allowed.",
          "title": "Context Sequences"
        }
      },
      "required": [
        "sequence"
      ],
      "title": "E1EncodeRequestItem",
      "type": "object"
    },
    "E1EncodeRequestParams": {
      "additionalProperties": false,
      "properties": {
        "repr_layers": {
          "description": "Hidden layers whose representations to return (negative indexes count from the last layer).",
          "items": {
            "type": "integer"
          },
          "title": "Repr Layers",
          "type": "array",
          "default": [
            -1
          ]
        },
        "include": {
          "description": "Optional outputs to compute and include in the response.",
          "items": {
            "$ref": "#/$defs/E1EncodeIncludeOptions"
          },
          "title": "Include",
          "type": "array",
          "default": [
            "mean"
          ]
        }
      },
      "title": "E1EncodeRequestParams",
      "type": "object"
    }
  },
  "additionalProperties": false,
  "properties": {
    "params": {
      "$ref": "#/$defs/E1EncodeRequestParams",
      "description": "Optional parameters controlling this action (defaults are used when omitted)."
    },
    "items": {
      "description": "Batch of inputs to process in a single request. Up to 8 sequences per request.",
      "items": {
        "$ref": "#/$defs/E1EncodeRequestItem"
      },
      "maxItems": 8,
      "minItems": 1,
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "E1EncodeRequest",
  "type": "object"
}

Response — E1EncodeResponse

Field	Type	Required	Constraints	Description
`results`	list[E1EncodeResponseResult]	yes		Per-input results, returned in the same order as the request items.

Nested types

E1EncodeResponseResult

Field	Type	Required	Description
`embeddings`	list[LayerEmbedding] \| null	no	Per-layer mean-pooled embedding vectors for the query sequence.
`residue_embeddings`	list[LayerPerTokenEmbeddings] \| null	no	Per-residue embedding vectors.
`logits`	list[list[number]] \| null	no	Per-position logits over the model vocabulary.
`vocab_tokens`	list[string] \| null	no	Vocabulary token order corresponding to the logits columns.
`context_sequence_count`	integer \| null	no	Number of context sequences used to condition this prediction.

LayerEmbedding

Field	Type	Required	Constraints	Description
`layer`	integer	yes		Model layer this representation was taken from.
`embedding`	list[number]	yes		Mean-pooled embedding vector for the sequence.

LayerPerTokenEmbeddings

Field	Type	Required	Constraints	Description
`layer`	integer	yes		Model layer this representation was taken from.
`embeddings`	list[list[number]]	yes		Per-token embedding vectors for the sequence at this layer.

Raw JSON Schema

{
  "$defs": {
    "E1EncodeResponseResult": {
      "description": "Response result for a single encoded sequence.",
      "exclude_none": true,
      "exclude_unset": true,
      "properties": {
        "embeddings": {
          "anyOf": [
            {
              "items": {
                "$ref": "#/$defs/LayerEmbedding"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Per-layer mean-pooled embedding vectors for the query sequence.",
          "title": "Embeddings"
        },
        "residue_embeddings": {
          "anyOf": [
            {
              "items": {
                "$ref": "#/$defs/LayerPerTokenEmbeddings"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Per-residue embedding vectors.",
          "title": "Residue Embeddings"
        },
        "logits": {
          "anyOf": [
            {
              "items": {
                "items": {
                  "type": "number"
                },
                "type": "array"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Per-position logits over the model vocabulary.",
          "title": "Logits"
        },
        "vocab_tokens": {
          "anyOf": [
            {
              "items": {
                "type": "string"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Vocabulary token order corresponding to the logits columns.",
          "title": "Vocab Tokens"
        },
        "context_sequence_count": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Number of context sequences used to condition this prediction.",
          "title": "Context Sequence Count"
        }
      },
      "title": "E1EncodeResponseResult",
      "type": "object"
    },
    "LayerEmbedding": {
      "description": "Embedding vector for a single layer.",
      "properties": {
        "layer": {
          "description": "Model layer this representation was taken from.",
          "title": "Layer",
          "type": "integer"
        },
        "embedding": {
          "description": "Mean-pooled embedding vector for the sequence.",
          "items": {
            "type": "number"
          },
          "title": "Embedding",
          "type": "array"
        }
      },
      "required": [
        "layer",
        "embedding"
      ],
      "title": "LayerEmbedding",
      "type": "object"
    },
    "LayerPerTokenEmbeddings": {
      "description": "Per-token embedding vectors for a single layer.",
      "properties": {
        "layer": {
          "description": "Model layer this representation was taken from.",
          "title": "Layer",
          "type": "integer"
        },
        "embeddings": {
          "description": "Per-token embedding vectors for the sequence at this layer.",
          "items": {
            "items": {
              "type": "number"
            },
            "type": "array"
          },
          "title": "Embeddings",
          "type": "array"
        }
      },
      "required": [
        "layer",
        "embeddings"
      ],
      "title": "LayerPerTokenEmbeddings",
      "type": "object"
    }
  },
  "properties": {
    "results": {
      "description": "Per-input results, returned in the same order as the request items.",
      "items": {
        "$ref": "#/$defs/E1EncodeResponseResult"
      },
      "title": "Results",
      "type": "array"
    }
  },
  "required": [
    "results"
  ],
  "title": "E1EncodeResponse",
  "type": "object"
}

`predict`¶

Call it

curl -X POST http://127.0.0.1:8000/api/v1/e1-150m/predict \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {
      "sequence": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE"
    }
  ]
}'

Request — E1PredictRequest

Field	Type	Required	Constraints	Description
`items`	list[E1PredictRequestItem]	yes	items 1–8	Batch of inputs to process in a single request. Up to 8 sequences per request.

Nested types

E1PredictRequestItem

Field	Type	Required	Constraints	Description
`sequence`	string	yes	len 1–2048	A protein sequence with one or more '?' mask tokens marking positions to predict.
`context_sequences`	list[string] \| null	no		Optional homologous sequences for context; no '?' mask tokens allowed in context sequences.

Raw JSON Schema

{
  "$defs": {
    "E1PredictRequestItem": {
      "additionalProperties": false,
      "description": "Single item for masked prediction.\n\nArgs:\n    sequence: Sequence with '?' tokens marking positions to predict.\n        Only the query sequence may contain mask tokens.\n    context_sequences: Optional list of homologous sequences for context.\n        Context sequences must NOT contain '?' mask tokens.",
      "properties": {
        "sequence": {
          "description": "A protein sequence with one or more '?' mask tokens marking positions to predict.",
          "maxLength": 2048,
          "minLength": 1,
          "title": "Sequence",
          "type": "string"
        },
        "context_sequences": {
          "anyOf": [
            {
              "items": {
                "type": "string"
              },
              "maxItems": 50,
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Optional homologous sequences for context; no '?' mask tokens allowed in context sequences.",
          "title": "Context Sequences"
        }
      },
      "required": [
        "sequence"
      ],
      "title": "E1PredictRequestItem",
      "type": "object"
    }
  },
  "additionalProperties": false,
  "properties": {
    "items": {
      "description": "Batch of inputs to process in a single request. Up to 8 sequences per request.",
      "items": {
        "$ref": "#/$defs/E1PredictRequestItem"
      },
      "maxItems": 8,
      "minItems": 1,
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "E1PredictRequest",
  "type": "object"
}

Response — E1PredictResponse

Field	Type	Required	Constraints	Description
`results`	list[E1PredictResponseResult]	yes		Per-input results, returned in the same order as the request items.

Nested types

E1PredictResponseResult

Field	Type	Required	Description
`logits`	list[list[number]]	yes	Per-position logits over the model vocabulary.
`sequence_tokens`	list[string]	yes	Per-position input tokens, aligned with the logits.
`vocab_tokens`	list[string]	yes	Vocabulary token order corresponding to the logits columns.

Raw JSON Schema

{
  "$defs": {
    "E1PredictResponseResult": {
      "properties": {
        "logits": {
          "description": "Per-position logits over the model vocabulary.",
          "items": {
            "items": {
              "type": "number"
            },
            "type": "array"
          },
          "title": "Logits",
          "type": "array"
        },
        "sequence_tokens": {
          "description": "Per-position input tokens, aligned with the logits.",
          "items": {
            "type": "string"
          },
          "title": "Sequence Tokens",
          "type": "array"
        },
        "vocab_tokens": {
          "description": "Vocabulary token order corresponding to the logits columns.",
          "items": {
            "type": "string"
          },
          "title": "Vocab Tokens",
          "type": "array"
        }
      },
      "required": [
        "logits",
        "sequence_tokens",
        "vocab_tokens"
      ],
      "title": "E1PredictResponseResult",
      "type": "object"
    }
  },
  "properties": {
    "results": {
      "description": "Per-input results, returned in the same order as the request items.",
      "items": {
        "$ref": "#/$defs/E1PredictResponseResult"
      },
      "title": "Results",
      "type": "array"
    }
  },
  "required": [
    "results"
  ],
  "title": "E1PredictResponse",
  "type": "object"
}

`log_prob`¶

Call it

curl -X POST http://127.0.0.1:8000/api/v1/e1-150m/log_prob \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {
      "sequence": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE"
    }
  ]
}'

Request — E1LogProbRequest

Field	Type	Required	Constraints	Description
`items`	list[E1LogProbRequestItem]	yes	items 1–8	Batch of inputs to process in a single request. Up to 8 sequences per request.

Nested types

E1LogProbRequestItem

Field	Type	Required	Constraints	Description
`sequence`	string	yes	len 1–2048	A protein sequence in the 20 canonical amino-acid codes to score.
`context_sequences`	list[string] \| null	no		Optional homologous sequences to condition scoring via block-causal attention; up to 50 sequences allowed.

Raw JSON Schema

{
  "$defs": {
    "E1LogProbRequestItem": {
      "additionalProperties": false,
      "description": "Single item for log probability prediction.\n\nArgs:\n    sequence: The query sequence to score. Must contain only the 20\n        canonical amino acids (ACDEFGHIKLMNPQRSTVWY).\n    context_sequences: Optional list of homologous sequences for context.\n        When provided, the model conditions on these sequences using\n        block-causal attention, typically improving fitness predictions.\n        Context sequences must also use only canonical amino acids.",
      "properties": {
        "sequence": {
          "description": "A protein sequence in the 20 canonical amino-acid codes to score.",
          "maxLength": 2048,
          "minLength": 1,
          "title": "Sequence",
          "type": "string"
        },
        "context_sequences": {
          "anyOf": [
            {
              "items": {
                "type": "string"
              },
              "maxItems": 50,
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Optional homologous sequences to condition scoring via block-causal attention; up to 50 sequences allowed.",
          "title": "Context Sequences"
        }
      },
      "required": [
        "sequence"
      ],
      "title": "E1LogProbRequestItem",
      "type": "object"
    }
  },
  "additionalProperties": false,
  "properties": {
    "items": {
      "description": "Batch of inputs to process in a single request. Up to 8 sequences per request.",
      "items": {
        "$ref": "#/$defs/E1LogProbRequestItem"
      },
      "maxItems": 8,
      "minItems": 1,
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "E1LogProbRequest",
  "type": "object"
}

Response — E1LogProbResponse

Field	Type	Required	Constraints	Description
`results`	list[E1LogProbResponseResult]	yes		Per-input results, returned in the same order as the request items.

Nested types

E1LogProbResponseResult

Field	Type	Required	Constraints	Description
`log_prob`	number	yes		Pseudo-log-likelihood of the sequence under the model.

Raw JSON Schema

{
  "$defs": {
    "E1LogProbResponseResult": {
      "properties": {
        "log_prob": {
          "description": "Pseudo-log-likelihood of the sequence under the model.",
          "title": "Log Prob",
          "type": "number"
        }
      },
      "required": [
        "log_prob"
      ],
      "title": "E1LogProbResponseResult",
      "type": "object"
    }
  },
  "properties": {
    "results": {
      "description": "Per-input results, returned in the same order as the request items.",
      "items": {
        "$ref": "#/$defs/E1LogProbResponseResult"
      },
      "title": "Results",
      "type": "array"
    }
  },
  "required": [
    "results"
  ],
  "title": "E1LogProbResponse",
  "type": "object"
}

Usage¶

One-line summary: Retrieval-augmented protein encoder language model that produces embeddings, masked predictions, and log-probability scores, optionally conditioned on homologous context sequences.

Overview¶

E1 is a protein language model developed by Profluent Bio and Synthyra. It extends the masked language modeling paradigm with retrieval-augmented inference: users can provide homologous sequences as context, and the model uses block-causal attention to condition predictions on evolutionary information without requiring explicit MSA computation.

E1 is available in three size variants (150M, 300M, 600M parameters) and supports three actions: embedding extraction (encode), masked token prediction (predict), and sequence log-probability scoring (log_prob). All actions support optional context sequences for improved accuracy.

Architecture¶

Property	Value
Architecture	Transformer encoder (masked LM with block-causal attention)
Training objective	Masked language modeling
Max sequence length	2048 tokens
Max context sequences	50
Mask token	`?`
License	Apache-2.0

Capabilities & Limitations¶

CAN be used for: - Generating per-residue and mean-pooled sequence embeddings - Masked token prediction (fill-in-the-blank with ? tokens) - Zero-shot variant effect prediction via log-probability scoring - Retrieval-augmented inference with homologous context sequences - Extracting logits restricted to the 20 canonical amino acids

CANNOT be used for: - Sequence generation or design (encoder-only model) - 3D structure prediction - Non-protein molecules (DNA, RNA, small molecules) - Multi-chain complex modeling

Other considerations: - Mask token is ? (not <mask> used by ESM models) - GPU memory snapshots are disabled due to SIGSEGV on restore - torch.compile is disabled due to flex_attention compilation errors - Batch size capped at 8 sequences per request - Context sequences are limited to 50 per item

Usage Examples¶

# Encode with context sequences
from models.e1.schema import (
    E1EncodeRequest,
    E1EncodeRequestItem,
    E1EncodeRequestParams,
)

encode_request = E1EncodeRequest(
    params=E1EncodeRequestParams(repr_layers=[-1], include=["mean"]),
    items=[
        E1EncodeRequestItem(
            sequence="TPSSKEMMSQALKATFSGFTKEQQRLGIPKDPRQWTETHVRDWVMWAVNEFSLKGVDFQKF",
            context_sequences=[
                "MPSSKEMMSQALKATFSGFTKEQQRLGIPKDPRQWTETHVRDWVMWAVNEFSLKGVDFQKL",
                "TPSSKELMSQALKATFSGFTKEQQRLGIPKDPRQWTETHVRDWVMWAVNEFSLKGVDFQKY",
            ],
        ),
    ],
)

# Predict masked positions
from models.e1.schema import E1PredictRequest, E1PredictRequestItem

predict_request = E1PredictRequest(
    items=[
        E1PredictRequestItem(
            sequence="TPSSKE?MSQALKAT?SGFTKEQQ",
        )
    ],
)

# Score log probability with context
from models.e1.schema import E1LogProbRequest, E1LogProbRequestItem

log_prob_request = E1LogProbRequest(
    items=[
        E1LogProbRequestItem(
            sequence="TPSSKEMMSQALKATFSGFTKEQQRLGIPKDPRQWTETHVRDWVMWAVNEFSLKGVDFQKF",
            context_sequences=["MPSSKEMMSQALKATFSGFTKEQQRLGIPKDPRQWTETHVRDWVMWAVNEFSLKGVDFQKL"],
        )
    ],
)

Architecture & training¶

Architecture¶

Model Type & Innovation¶

E1 is an encoder protein language model developed by Profluent Bio and Synthyra. It is a masked language model based on a transformer encoder architecture with a novel feature: retrieval-augmented inference. E1 supports conditioning on homologous context sequences via block-causal attention, enabling improved predictions when evolutionary context is available.

The key innovation is the multi-sequence input format: context sequences (homologs) are prepended to the query sequence, and the model uses block-causal attention so that the query attends to both itself and the context, while context sequences only attend to themselves. This simulates the information in a multiple sequence alignment (MSA) without requiring explicit MSA construction.

E1 uses ? as the mask token (rather than <mask> used by ESM models).

Parameters & Layers¶

Variant	Parameters	Layers	GPU	Dtype
E1-150M	150M	20	T4	float16
E1-300M	300M	20	L4	bfloat16
E1-600M	600M	30	L4	bfloat16

From Jain et al. (2025): E1 models use a standard Transformer encoder architecture augmented with block-causal attention for retrieval augmentation. Global block-causal attention is used every three layers, while all other layers use intra-sequence attention. All models use Rotary Position Embedding (RoPE). E1 150M has 20 layers, E1 300M has 20 layers, and E1 600M has 30 layers. Models were trained on 4 trillion tokens (batch size 2^20 tokens) from the Profluent Protein Atlas (PPA-1) and UniRef Version 2411 datasets, using a warmup-stable-decay learning rate schedule with Stable AdamW optimizer. E1 600M was trained on 64 H100 GPUs for 25 days. Training used curriculum learning, gradually increasing total length (8192 to 32768 tokens) and number of sequences per instance (2 to 512). PPA-1 was used exclusively for the first 1.5 trillion tokens, then mixed with UniRef in a 60:40 ratio.

Common across all variants:

Property	Value
Max sequence length	2048 tokens
Max context sequences	50
Mask token	`?`
Tokenizer	Custom E1 tokenizer
Multi-sequence format	Comma-separated (context sequences, then query)
Tokenization per sequence	`<bos> 1 SEQUENCE 2 <eos>`

Training Data¶

Property	Details
Source	Profluent Protein Atlas (PPA-1) and UniRef Version 2411 (PPA-1 only for first 1.5T tokens, then 60:40 PPA-1/UniRef; 4 trillion total training tokens)
Training objective	Masked language modeling with block-causal attention

Loss Function & Objective¶

Masked language modeling (MLM) with cross-entropy loss, enhanced by block-causal attention that enables retrieval-augmented training where context sequences inform the query prediction.

Tokenization / Input Processing¶

Property	Details
Encode sequences	Extended amino acid alphabet (20 standard + B, X, Z, U, O)
Predict sequences	Extended AA + `?` mask token (one or more `?` required)
Predict log prob sequences	20 canonical amino acids only
Context sequences	Optional list of homologs (max 50, no `?` tokens allowed)
Multi-sequence format	`CONTEXT1,CONTEXT2,...,QUERY`
Per-sequence tokenization	`<bos> 1 SEQUENCE 2 <eos>` (4 overhead tokens per sequence: `<bos>`, `1`, `2`, `<eos>`)

Performance & Benchmarks¶

Published Benchmarks¶

From Jain et al. (2025), Table 1 -- ProteinGym v1.3 (217 DMS substitution assays), average Spearman correlation:

Model	Mode	Avg Spearman	NDCG@10
ESM2-150M	Single sequence	0.387	0.729
ESM2-650M	Single sequence	0.414	0.747
ESMC-600M	Single sequence	0.405	0.746
E1 150M	Single sequence	0.401	0.744
E1 300M	Single sequence	0.416	0.748
E1 600M	Single sequence	0.420	0.749
PoET	+ Homologs	0.470	0.784
E1 150M	+ Homologs	0.473	0.785
E1 300M	+ Homologs	0.475	0.787
E1 600M	+ Homologs	0.477	0.788

E1 outperforms all ESM-2 and ESMC family models in single-sequence mode at comparable sizes. With retrieval augmentation, E1 achieves state-of-the-art performance among publicly available models, surpassing PoET and MSA Pairformer. E1 also achieves superior unsupervised contact-map prediction (Precision@L) on CAMEO subsets.

BioLM Verification Results¶

Test Case	Tolerance	Status
Encode (mean, single sequence)	rel_tol 1e-4, cosine < 0.02	PASS
Encode (per-token, single sequence, layer 15)	rel_tol 1e-4, cosine < 0.02	PASS
Encode (with context sequences)	rel_tol 1e-4, cosine < 0.02	PASS
Predict (masked tokens)	rel_tol 1e-4, cosine < 0.02	PASS
Predict log prob (single)	Negative finite float	PASS
Predict log prob (with context)	Negative finite float	PASS

Comparison to Alternatives¶

Model	Type	Key Advantage	Key Disadvantage
E1 (this)	Retrieval-augmented MLM	Context sequences improve predictions without MSA	Newer, fewer benchmarks
ESM2	Standard MLM	Most widely adopted, proven embeddings	No retrieval augmentation
SaProt	Structure-aware MLM	Uses structure tokens	Requires structure input
MSA Transformer	MSA-based	Direct MSA processing	Requires explicit MSA computation
ProtTrans (ProtT5)	Encoder-decoder	Generation capability	Larger compute footprint

Error Bars & Confidence¶

E1 is deterministic when seeds are set. The same input with the same context sequences produces the same output. Adding or removing context sequences will change the output (by design).

GPU-specific note: torch.compile / torch._dynamo is disabled due to flex_attention compilation errors on PyTorch 2.6.0. This means eager mode is used, which is slightly slower but avoids segmentation faults.

Strengths & Limitations¶

Pros¶

Retrieval-augmented inference via context sequences (up to 50 homologs)
Three size variants (150M, 300M, 600M) for speed/quality tradeoff
Multiple output types: mean embeddings, per-token, logits, log probabilities
No MSA computation needed -- just provide homologous sequences as context
Apache-2.0 license with no restrictions

Cons¶

Newer model with fewer published benchmarks than ESM2
torch.compile disabled (eager mode only) due to PyTorch 2.6.0 compatibility
No GPU memory snapshots (causes SIGSEGV on restore) -- slower cold starts
? mask token differs from ESM convention (<mask>)
Logits restricted to 20 canonical amino acids only

Known Failure Modes¶

GPU snapshot SIGSEGV: Memory snapshots cause segfaults; disabled entirely
torch.compile errors: flex_attention compilation fails with sympy Relational errors; dynamo disabled
Context sequence mask tokens: Context sequences containing ? are rejected (only query may have masks)
Very long multi-sequence inputs: Total token count (all context + query + overhead tokens) may exceed memory for many long context sequences

Implementation Details¶

Inference Pipeline¶

Encode pipeline:

Request
  |-- 1. Validate sequences (extended AA alphabet)
  |-- 2. Build multi-sequence input (context + query, comma-separated)
  |-- 3. Calculate query token positions
  |-- 4. Forward pass with output_hidden_states=True
  |-- 5. Extract query positions from hidden states
  |-- 6. Post-process: mean pooling, per-token, or logits
  |-- 7. Return E1EncodeResponse

Predict pipeline:

Request
  |-- 1. Validate sequences (extended AA + `?`)
  |-- 2. Build multi-sequence input
  |-- 3. Forward pass -> logits
  |-- 4. Extract query positions, slice to canonical 20 AA
  |-- 5. Return E1PredictResponse

Predict log prob pipeline:

Request
  |-- 1. Validate sequences (canonical 20 AA only)
  |-- 2. Build multi-sequence input
  |-- 3. Forward pass -> logits
  |-- 4. For each query position: log-softmax over canonical 20 AA
  |-- 5. Sum log P(residue_i) across all positions
  |-- 6. Return E1LogProbResponse

Memory & Compute Profile¶

Variant	GPU	VRAM	Dtype	Inference Latency
E1-150M	T4 (16 GB)	~4 GB	float16	~100ms/seq
E1-300M	L4 (24 GB)	~8 GB	bfloat16	~150ms/seq
E1-600M	L4 (24 GB)	~12 GB	bfloat16	~250ms/seq

Determinism & Reproducibility¶

Setting	Value
`torch.manual_seed`	42
`torch.cuda.manual_seed_all`	42
`torch._dynamo.config.disable`	True
`torch.no_grad`	Yes (inference)
Memory snapshots	Disabled (SIGSEGV on restore)

Caching Behavior¶

Response caching is handled by the serving layer, not by the model container. Cache keys include context sequences when provided.

Versions & Changelog¶

Version	Date	Changes
v1	2025-12-04	Initial implementation with encode, predict, and log_prob actions
v1 (updated)	2025-12-04	Added context_sequences support (retrieval-augmented mode)
v1 (updated)	2025-12-11	Disabled torch.compile/dynamo to fix flex_attention errors
v1 (updated)	2025-12-13	Disabled GPU memory snapshots to fix SIGSEGV

Biology¶

Molecule Coverage¶

Primary Molecule Type(s)¶

E1 operates on protein sequences and supports retrieval-augmented inference where homologous context sequences improve prediction quality. It accepts sequences using the extended amino acid alphabet (20 standard + B, X, Z, U, O).

Performance characteristics by protein type:

Globular proteins: Primary use case. Both embeddings and masked predictions benefit from evolutionary context provided via context sequences.
Enzymes: Well-suited for active-site analysis via masked prediction at catalytic positions.
Protein families with known homologs: E1 excels when context sequences (homologs) are provided, simulating MSA information without explicit alignment.
Orphan proteins: Can be encoded without context, but predictions may be less accurate than when homologs are available.
Antibodies: Variable regions can be analyzed; providing germline or related antibody sequences as context may improve CDR predictions.
Peptides: Short sequences provide limited context; consider peptide-specific models.

Cross-Applicability¶

Molecule Type	Applicability	Evidence	Caveats
Globular proteins	High	Primary design target	Context sequences improve quality
Enzymes	High	Active-site residues well-captured by MLM	No explicit catalytic mechanism
Protein families	High	Context sequences simulate MSA	Up to 50 context sequences supported
Antibodies	Moderate	Can analyze variable regions	Not antibody-specific trained
Orphan proteins	Moderate	Works without context, lower accuracy	No evolutionary context available
Peptides	Low	Short sequences lack context	Use peptide-specific models
DNA/RNA	Not supported	Not in training data	Protein sequences only

Biological Problems Addressed¶

Retrieval-Augmented Protein Representation¶

Problem: Traditional protein language models (e.g., ESM2) process sequences in isolation. Important evolutionary information captured in multiple sequence alignments (MSAs) is lost unless explicitly computed -- and MSA computation is slow (minutes to hours per query).

How E1 helps: The encode action accepts optional context_sequences (homologous sequences) that are processed jointly with the query using block-causal attention. The query sequence attends to all context, gaining evolutionary context without explicit MSA construction.

Biological meaning: Context sequences provide the model with information about which positions are conserved (functionally important) and which are variable (tolerant to substitution) across related proteins. This is the same information encoded in MSAs but provided in a more flexible format -- any set of related sequences works, without requiring formal alignment.

Zero-Shot Fitness Prediction¶

Problem: Predicting whether a mutation improves or degrades protein function is critical for protein engineering and disease variant interpretation. Experimental testing is expensive.

How E1 helps: The log_prob action computes the total log probability of a sequence. By comparing log P(wild-type) vs log P(mutant), researchers can estimate mutation effects. When context sequences are provided, the model conditions on evolutionary information, typically improving fitness predictions.

Biological meaning: A large drop in log probability (mutant much less likely than wild-type) suggests the mutation disrupts an evolutionarily conserved position and is likely deleterious. Context sequences sharpen this signal by providing explicit examples of what evolution "expects" at each position.

Masked Sequence Prediction¶

Problem: Determining which amino acids are compatible at specific positions in a protein, accounting for both local and evolutionary context.

How E1 helps: The predict action takes sequences with ? mask tokens and returns logits over the 20 canonical amino acids at each masked position. Context sequences can be provided to improve predictions.

Biological meaning: The predicted distribution at each masked position reflects evolutionary constraints. Highly constrained positions (active sites, structural cores) will have narrow distributions; tolerant positions (surface loops) will have broad distributions. This guides rational mutagenesis and sequence design.

Protein Embedding for Downstream Tasks¶

Problem: Numerical representations of proteins are needed for machine learning tasks such as function prediction, clustering, and similarity search.

How E1 helps: The encode action produces mean-pooled, per-token, or logit-based representations from specified transformer layers. Multiple layers can be requested simultaneously.

Biological meaning: E1 embeddings capture evolutionary, structural, and biophysical properties. Proteins with similar function or structure have similar embeddings, enabling clustering, nearest-neighbor search, and feature extraction for downstream classifiers.

Applied Use Cases¶

E1 is a recent model (2025). Anticipated use cases include:

Variant effect prediction: Scoring mutations using log_prob with context sequences for improved accuracy
Sequence design: Using masked prediction to explore compatible amino acids at design positions
Protein family analysis: Encoding protein families with shared context for consistent embeddings
Antibody humanization: Using germline sequences as context when predicting at CDR positions
Feature extraction: Using E1 embeddings as input to downstream stability, function, or localization classifiers

Complementary Models¶

ESM2 (this catalog): E1 provides retrieval-augmented embeddings as an alternative to ESM2's single-sequence approach. Use ESM2 when no homologs are available; use E1 when context sequences can be provided.
SPURS (this catalog): Can use E1 embeddings or log-prob scores alongside structure-based ddG predictions.
ESMFold / Chai-1 (this catalog): Structure prediction models that can validate E1-guided sequence designs.

Alternative Models¶

Alternative	Advantage Over E1	Disadvantage vs E1
ESM2	Most widely adopted, proven benchmarks	No retrieval augmentation
MSA Transformer	Direct MSA processing	Requires explicit (slow) MSA computation
SaProt	Structure tokens improve representations	Requires structure input
ESM3	Multimodal (sequence + structure + function)	Larger resource requirements
ProtTrans (ProtT5)	Encoder-decoder, generation capable	Larger compute footprint

When to choose E1: Use E1 when you have homologous sequences available and want to improve embedding/prediction quality without computing a full MSA. E1 is also a good choice when you need log-probability scoring with evolutionary context.

When to choose alternatives: Use ESM2 when homologs are not available or for compatibility with existing pipelines; use MSA Transformer when a high-quality MSA is already computed; use SaProt when structure is available.

Biological Background¶

Protein language models learn the statistical patterns of amino acid sequences from large protein databases. By training on millions of sequences from diverse organisms, they implicitly learn which amino acids are compatible at each position given the surrounding context -- capturing evolutionary conservation, structural constraints, and biophysical properties.

Retrieval-augmented inference: Traditional protein LMs process sequences independently. E1 introduces a retrieval-augmented approach where related sequences (homologs) are provided as context. The model uses block-causal attention: the query sequence can attend to all context sequences, while context sequences only attend to themselves. This provides the query with information about evolutionary conservation without requiring formal sequence alignment.

Block-causal attention: An attention pattern where different parts of the input have different visibility. In E1, context sequences form independent blocks (each only sees itself), while the query sequence can see all blocks. This simulates the effect of reading across an MSA column -- the query "sees" what amino acids appear at corresponding positions in related proteins.

Key terminology: - Context sequences: Homologous protein sequences provided alongside the query to improve predictions. These serve as an implicit MSA. - Block-causal attention: Attention pattern where query attends to all sequences but context sequences only attend to themselves. - Log probability / pseudo-log-likelihood: A score measuring how "expected" a sequence is under the model. Higher (less negative) values indicate more natural sequences. - Masked language modeling: Training objective where random positions are masked and the model predicts the original amino acid. - MSA (Multiple Sequence Alignment): An alignment of multiple related protein sequences that reveals patterns of conservation and variation. E1's context sequences provide similar information without requiring explicit alignment. - Homolog: A protein related by common evolutionary ancestry. Orthologs (same gene, different species) and paralogs (gene duplication) are both useful as context sequences.

Sources & license¶

License: Apache-2.0 (text) — Encoder protein language model by Profluent/Synthyra

Papers

E1: Retrieval-Augmented Protein Encoder Models — bioRxiv preprint, 2025 · DOI

Source repositories

huggingface: https://huggingface.co/Synthyra/Profluent-E1-150M

E1¶

At a glance¶

API & schema¶

encode¶

predict¶

log_prob¶

Usage¶

Overview¶

Architecture¶

Capabilities & Limitations¶

Usage Examples¶

Architecture & training¶

Architecture¶

Model Type & Innovation¶

Parameters & Layers¶

Training Data¶

Loss Function & Objective¶

Tokenization / Input Processing¶

Performance & Benchmarks¶

Published Benchmarks¶

BioLM Verification Results¶

Comparison to Alternatives¶

Error Bars & Confidence¶

Strengths & Limitations¶

Pros¶

Cons¶

Known Failure Modes¶

Implementation Details¶

Inference Pipeline¶

Memory & Compute Profile¶

Determinism & Reproducibility¶

Caching Behavior¶

Versions & Changelog¶

Biology¶

Molecule Coverage¶

Primary Molecule Type(s)¶

Cross-Applicability¶

Biological Problems Addressed¶

Retrieval-Augmented Protein Representation¶

Zero-Shot Fitness Prediction¶

Masked Sequence Prediction¶

Protein Embedding for Downstream Tasks¶

Applied Use Cases¶

Related Models¶

Complementary Models¶

Alternative Models¶

Biological Background¶

Sources & license¶

`encode`¶

`predict`¶

`log_prob`¶