ZymCTRL¶

EC number-conditioned GPT-2 language model for generating novel enzyme sequences with targeted catalytic function.

License: Apache-2.0 · Molecules: protein · Tasks: sequence_generation, embedding

Actions: generate, encode · Variants: 1

At a glance¶

Use it when

You need to design novel enzymes with a specific catalytic function and know the target EC number
You are exploring enzyme sequence space for a given EC class and need diverse candidates (~53% average identity to natural enzymes) for experimental screening
You need function-aware enzyme embeddings that capture catalytic classification rather than just sequence similarity
You are building an enzyme engineering pipeline where candidates must be ranked by perplexity-based fitness scores before experimental validation
You are designing enzymes for industrial biocatalysis, pharmaceutical synthesis, environmental remediation, or carbon capture and need computationally generated starting points
You need zero-shot enzyme generation for a specific EC class without fine-tuning — the model covers all seven major EC classes out of the box
You are fine-tuning a generative model for a specific enzyme family and need a pre-trained enzyme-specialized starting point

Strengths

EC-number conditioned generation enables targeted enzyme design — the only model in this catalog that directly accepts an enzymatic function specification (EC number) as a generation control tag
Trained on 37M enzyme sequences from UniProt with EC annotations, providing dense coverage of the enzyme classification hierarchy across all seven major EC classes
Experimentally validated: generated carbonic anhydrases (EC 4.2.1.1) showed catalytic activity, demonstrating the model produces functional enzymes, not just statistically plausible sequences
Dual-action API (generate + encode) provides both enzyme generation and enzyme-function-aware embeddings in a single deployment
Perplexity-based filtering (threshold <1.75) provides a practical quality metric for ranking generated candidates before experimental testing
Fine-tuning capability demonstrated on lactate dehydrogenase (EC 1.1.1.27), enabling specialization for specific EC classes when deeper coverage is needed
Generated sequences average ~53% identity to natural proteins — novel enough to explore new sequence space while maintaining fold-level similarity to natural enzymes
Apache-2.0 license permits unrestricted commercial and academic use for enzyme engineering applications

Limitations

Enzyme-only model — cannot generate non-enzyme proteins; antibodies, structural proteins, binding proteins, and other non-catalytic proteins are outside its training distribution
Single-chain generation only — cannot design multi-subunit enzyme complexes or model intersubunit interactions
Maximum sequence length of ~1000 amino acids (1024 token limit includes EC number and control tokens) excludes very large enzymes
No structure awareness — generates sequences without 3D geometric constraints; generated enzymes must be structurally validated with separate tools (ESMFold, Chai-1)
Performance varies by EC class representation in training data — rare or recently classified EC numbers may produce lower-quality sequences
738M parameters on a T4 GPU is moderate cost; not as lightweight as CPU-only options for scoring tasks
Cannot model enzymes requiring post-translational modifications (glycosylation, phosphorylation) or metal cofactors for activity

Reach for something else when

You need to generate non-enzyme proteins such as antibodies, structural proteins, or binding proteins (use progen2 for general proteins or the OAS variant for antibodies)
You need structure-conditioned sequence design from a target backbone (use mpnn for comprehensive inverse folding or esm_if1 for lightweight inverse folding)
You need protein embeddings for general downstream ML tasks beyond enzyme function (use esm2, which provides broader protein representations)
You need to design multi-subunit enzyme complexes with modeled intersubunit contacts
You need to generate sequences longer than ~1000 amino acids
You are designing proteins where the desired function is not well-described by an EC number (e.g., binding, transport, signaling) — use ProGen2 or DSM instead
You need masked infilling to design specific enzyme regions while fixing the scaffold (use dsm, which supports masked diffusion)
You need protein-protein interaction-aware enzyme design (use dsm with the PPI variant)

Alternatives

Model	Better when	Worse when
progen2	Four variants covering general, antibody (OAS), and metagenomic proteins; broader protein-type coverage beyond enzymes; bidirectional log-likelihood scoring.	No EC-number conditioning — cannot target a specific catalytic function; not enzyme-specialized; zymctrl's generated enzymes are experimentally validated for activity.
dsm	Masked diffusion enables bidirectional generation and infilling plus embeddings and scoring; PPI variant for interaction-aware design.	No function-conditioning mechanism; zymctrl's EC conditioning directly targets catalytic function, which DSM cannot replicate.
esm2	Best general-purpose protein embeddings and fitness scoring; five size variants; widest ecosystem compatibility.	Encoder-only — cannot generate enzyme sequences; no function conditioning; overlaps zymctrl only on the encode/embedding action.

API & schema¶

Call an action with POST /api/v1/{slug}/{action} — the request envelope is {"items": [...], "params": {...}} and a success returns {"results": [...]}. See the HTTP API page for the base URL, error shape, and full contract.

`generate`¶

Call it

curl -X POST http://127.0.0.1:8000/api/v1/zymctrl/generate \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {
      "ec_number": "string"
    }
  ]
}'

Request — ZymCTRLGenerateRequest

Field	Type	Required	Constraints	Description
`params`	ZymCTRLGenerateParams	no		Optional parameters controlling this action (defaults are used when omitted).
`items`	list[ZymCTRLGenerateRequestItem]	yes	items 1–1	Batch of inputs to process in a single request. Up to 1 EC number per request.

Nested types

ZymCTRLGenerateParams

Field	Type	Required	Constraints	Description
`seed`	integer \| null	no		Random seed for reproducible sampling.
`temperature`	number	no	>0.0; ≤2.0; default `0.8`	Sampling temperature; higher values increase diversity.
`top_k`	integer	no	≥1; ≤50; default `9`	Top-k sampling cutoff; only the k most likely tokens are sampled.
`repetition_penalty`	number	no	≥1.0; ≤2.0; default `1.2`	Penalty applied to previously generated tokens to discourage repetitive sequences; higher values reduce repetition.
`num_samples`	integer	no	≥1; ≤20; default `5`	Number of sequences to generate per input.
`max_length`	integer	no	≥50; ≤1024; default `256`	Maximum length of the generated sequence.

ZymCTRLGenerateRequestItem

Field	Type	Required	Constraints	Description
`ec_number`	string	yes		Enzyme Commission (EC) number conditioning generation (e.g. '1.1.1.1').

Raw JSON Schema

{
  "$defs": {
    "ZymCTRLGenerateParams": {
      "additionalProperties": false,
      "properties": {
        "seed": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Random seed for reproducible sampling.",
          "title": "Seed"
        },
        "temperature": {
          "default": 0.8,
          "description": "Sampling temperature; higher values increase diversity.",
          "exclusiveMinimum": 0.0,
          "maximum": 2.0,
          "title": "Temperature",
          "type": "number"
        },
        "top_k": {
          "default": 9,
          "description": "Top-k sampling cutoff; only the k most likely tokens are sampled.",
          "maximum": 50,
          "minimum": 1,
          "title": "Top K",
          "type": "integer"
        },
        "repetition_penalty": {
          "default": 1.2,
          "description": "Penalty applied to previously generated tokens to discourage repetitive sequences; higher values reduce repetition.",
          "maximum": 2.0,
          "minimum": 1.0,
          "title": "Repetition Penalty",
          "type": "number"
        },
        "num_samples": {
          "default": 5,
          "description": "Number of sequences to generate per input.",
          "maximum": 20,
          "minimum": 1,
          "title": "Num Samples",
          "type": "integer"
        },
        "max_length": {
          "default": 256,
          "description": "Maximum length of the generated sequence.",
          "maximum": 1024,
          "minimum": 50,
          "title": "Max Length",
          "type": "integer"
        }
      },
      "title": "ZymCTRLGenerateParams",
      "type": "object"
    },
    "ZymCTRLGenerateRequestItem": {
      "additionalProperties": false,
      "properties": {
        "ec_number": {
          "description": "Enzyme Commission (EC) number conditioning generation (e.g. '1.1.1.1').",
          "title": "Ec Number",
          "type": "string"
        }
      },
      "required": [
        "ec_number"
      ],
      "title": "ZymCTRLGenerateRequestItem",
      "type": "object"
    }
  },
  "additionalProperties": false,
  "properties": {
    "params": {
      "$ref": "#/$defs/ZymCTRLGenerateParams",
      "description": "Optional parameters controlling this action (defaults are used when omitted)."
    },
    "items": {
      "description": "Batch of inputs to process in a single request. Up to 1 EC number per request.",
      "items": {
        "$ref": "#/$defs/ZymCTRLGenerateRequestItem"
      },
      "maxItems": 1,
      "minItems": 1,
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "ZymCTRLGenerateRequest",
  "type": "object"
}

Response — ZymCTRLGenerateResponse

Field	Type	Required	Constraints	Description
`results`	list[list[ZymCTRLGenerateResponseGenerated]]	yes		Per-input results, returned in the same order as the request items.

Nested types

ZymCTRLGenerateResponseGenerated

Field	Type	Required	Constraints	Description
`sequence`	string	yes		A protein sequence in single-letter amino-acid codes.
`perplexity`	number	yes		Perplexity of the sequence under the model (lower means more likely).

Raw JSON Schema

{
  "$defs": {
    "ZymCTRLGenerateResponseGenerated": {
      "properties": {
        "sequence": {
          "description": "A protein sequence in single-letter amino-acid codes.",
          "title": "Sequence",
          "type": "string"
        },
        "perplexity": {
          "description": "Perplexity of the sequence under the model (lower means more likely).",
          "title": "Perplexity",
          "type": "number"
        }
      },
      "required": [
        "sequence",
        "perplexity"
      ],
      "title": "ZymCTRLGenerateResponseGenerated",
      "type": "object"
    }
  },
  "properties": {
    "results": {
      "description": "Per-input results, returned in the same order as the request items.",
      "items": {
        "items": {
          "$ref": "#/$defs/ZymCTRLGenerateResponseGenerated"
        },
        "type": "array"
      },
      "title": "Results",
      "type": "array"
    }
  },
  "required": [
    "results"
  ],
  "title": "ZymCTRLGenerateResponse",
  "type": "object"
}

`encode`¶

Call it

curl -X POST http://127.0.0.1:8000/api/v1/zymctrl/encode \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {
      "sequence": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE"
    }
  ]
}'

Request — ZymCTRLEncodeRequest

Field	Type	Required	Constraints	Description
`params`	ZymCTRLEncodeParams	no		Optional parameters controlling this action (defaults are used when omitted).
`items`	list[ZymCTRLEncodeRequestItem]	yes	items 1–8	Batch of inputs to process in a single request. Up to 8 sequences per request.

Nested types

ZymCTRLEncodeParams

Field	Type	Required	Constraints	Description
`pooling`	ZymCTRLPoolingType	no	default `mean`	Embedding pooling strategy: mean (average over residues), last (final token), or per_residue (one embedding per residue).
`layer`	integer	no	≥-36; ≤36; default `-1`	Model layer this representation was taken from.

ZymCTRLEncodeRequestItem

Field	Type	Required	Constraints	Description
`sequence`	string	yes	len 1–1024	A protein sequence in single-letter amino-acid codes.
`ec_number`	string \| null	no		Optional EC number providing functional context for the embedding (e.g. '3.5.5.1').

ZymCTRLPoolingType

Allowed values: mean, last, per_residue, per_residue

Raw JSON Schema

{
  "$defs": {
    "ZymCTRLEncodeParams": {
      "additionalProperties": false,
      "properties": {
        "pooling": {
          "$ref": "#/$defs/ZymCTRLPoolingType",
          "default": "mean",
          "description": "Embedding pooling strategy: mean (average over residues), last (final token), or per_residue (one embedding per residue)."
        },
        "layer": {
          "default": -1,
          "description": "Model layer this representation was taken from.",
          "maximum": 36,
          "minimum": -36,
          "title": "Layer",
          "type": "integer"
        }
      },
      "title": "ZymCTRLEncodeParams",
      "type": "object"
    },
    "ZymCTRLEncodeRequestItem": {
      "additionalProperties": false,
      "properties": {
        "sequence": {
          "description": "A protein sequence in single-letter amino-acid codes.",
          "maxLength": 1024,
          "minLength": 1,
          "title": "Sequence",
          "type": "string"
        },
        "ec_number": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Optional EC number providing functional context for the embedding (e.g. '3.5.5.1').",
          "title": "Ec Number"
        }
      },
      "required": [
        "sequence"
      ],
      "title": "ZymCTRLEncodeRequestItem",
      "type": "object"
    },
    "ZymCTRLPoolingType": {
      "enum": [
        "mean",
        "last",
        "per_residue",
        "per_residue"
      ],
      "title": "ZymCTRLPoolingType",
      "type": "string"
    }
  },
  "additionalProperties": false,
  "properties": {
    "params": {
      "$ref": "#/$defs/ZymCTRLEncodeParams",
      "description": "Optional parameters controlling this action (defaults are used when omitted)."
    },
    "items": {
      "description": "Batch of inputs to process in a single request. Up to 8 sequences per request.",
      "items": {
        "$ref": "#/$defs/ZymCTRLEncodeRequestItem"
      },
      "maxItems": 8,
      "minItems": 1,
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "ZymCTRLEncodeRequest",
  "type": "object"
}

Response — ZymCTRLEncodeResponse

Field	Type	Required	Constraints	Description
`results`	list[ZymCTRLEncodeResponseResult]	yes		Per-input results, returned in the same order as the request items.

Nested types

ZymCTRLEncodeResponseResult

Field	Type	Required	Description
`sequence_index`	integer	yes	Index of the corresponding input sequence within the request batch.
`embedding`	list[number] \| null	no	Embedding vector for the sequence; returned when pooling is 'mean' or 'last'.
`residue_embeddings`	list[list[number]] \| null	no	Per-residue embedding vectors.

Raw JSON Schema

{
  "$defs": {
    "ZymCTRLEncodeResponseResult": {
      "exclude_none": true,
      "exclude_unset": true,
      "properties": {
        "sequence_index": {
          "description": "Index of the corresponding input sequence within the request batch.",
          "title": "Sequence Index",
          "type": "integer"
        },
        "embedding": {
          "anyOf": [
            {
              "items": {
                "type": "number"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Embedding vector for the sequence; returned when pooling is 'mean' or 'last'.",
          "title": "Embedding"
        },
        "residue_embeddings": {
          "anyOf": [
            {
              "items": {
                "items": {
                  "type": "number"
                },
                "type": "array"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Per-residue embedding vectors.",
          "title": "Residue Embeddings"
        }
      },
      "required": [
        "sequence_index"
      ],
      "title": "ZymCTRLEncodeResponseResult",
      "type": "object"
    }
  },
  "properties": {
    "results": {
      "description": "Per-input results, returned in the same order as the request items.",
      "items": {
        "$ref": "#/$defs/ZymCTRLEncodeResponseResult"
      },
      "title": "Results",
      "type": "array"
    }
  },
  "required": [
    "results"
  ],
  "title": "ZymCTRLEncodeResponse",
  "type": "object"
}

Usage¶

One-line summary: EC number-conditioned GPT-2 language model for generating novel enzyme sequences with targeted catalytic function.

Overview¶

ZymCTRL is a conditional protein language model developed by the AI4PD group (Noelia Ferruz lab) that generates enzyme sequences conditioned on Enzyme Commission (EC) numbers. Built on a GPT-2 architecture with 738M parameters, it was trained on 37 million enzyme sequences from UniProt annotated with EC classifications.

The key innovation is character-level tokenization of EC numbers (e.g., "2.7.1.1" becomes ["2", ".", "7", ".", "1", ".", "1"]), which enables the model to learn hierarchical relationships across the enzyme classification system and transfer knowledge between related catalytic functions. This allows zero-shot generation of enzymes for any EC class without fine-tuning.

ZymCTRL is the first open-source conditional language model specifically designed for enzyme generation. Generated sequences average approximately 53% identity to natural proteins, indicating genuinely novel designs rather than memorized training data. The model has been experimentally validated: generated carbonic anhydrases showed catalytic activity in wet-lab tests.

Architecture¶

Property	Value
Architecture	GPT-2 Transformer (decoder-only, autoregressive)
Parameters	738M
Layers	36
Hidden dimensions	1280
Training data	37M enzyme sequences from UniProt with EC annotations (July 2022)
Training compute	48 NVIDIA A100 GPUs, ~15,000 GPU hours
Max sequence length	1024 tokens

Key innovation: EC numbers are tokenized at the character level (e.g., "2.7.1.1" becomes ["2", ".", "7", ".", "1", ".", "1"]), enabling transfer learning across related enzyme classes.

Capabilities & Limitations¶

CAN be used for: - Zero-shot enzyme generation for any EC class (all four levels of the EC hierarchy) - Conditional sequence design for specific catalytic reactions - Sequence embedding extraction for enzyme similarity analysis - Exploring novel enzyme sequence space

CANNOT be used for: - Non-enzyme proteins (model trained exclusively on EC-annotated sequences) - Substrate-level specificity within an EC class (e.g., cannot specify which sugar a kinase acts on) - Sequences longer than ~1000 amino acids (1024 token limit includes EC and control tokens) - Structure prediction (use ESMFold or Chai-1 for structural validation of generated sequences) - Multi-chain enzyme complexes (generates single chains only)

Other considerations: - Generate outputs are stochastic: different runs produce different sequences. Use the seed parameter for reproducibility. - Broad-substrate EC classes (e.g., hexokinases EC 2.7.1.1) produce more heterogeneous outputs. Narrow-substrate classes yield more consistent results. - Recommended workflow: generate 100-1000 sequences, rank by perplexity, and keep the best ~5%. The model card recommends aiming for perplexity below ~1.5-1.75 (lower is better); the "best ~5%" is a separate selection heuristic, not a fixed perplexity cutoff. - Fine-tuning on a specific EC class can improve quality for that class (demonstrated in the paper for lactate dehydrogenase).

Usage Examples¶

# Generate enzyme sequences for carbonic anhydrase (EC 4.2.1.1)
from models.zymctrl.schema import (
    ZymCTRLGenerateParams,
    ZymCTRLGenerateRequest,
    ZymCTRLGenerateRequestItem,
)

generate_request = ZymCTRLGenerateRequest(
    params=ZymCTRLGenerateParams(
        temperature=0.8,
        top_k=9,
        repetition_penalty=1.2,
        num_samples=10,
        max_length=256,
        seed=42,  # For reproducibility
    ),
    items=[
        ZymCTRLGenerateRequestItem(ec_number="4.2.1.1"),
    ],
)

# Extract embeddings with EC context
from models.zymctrl.schema import (
    ZymCTRLEncodeParams,
    ZymCTRLEncodeRequest,
    ZymCTRLEncodeRequestItem,
    ZymCTRLPoolingType,
)

encode_request = ZymCTRLEncodeRequest(
    params=ZymCTRLEncodeParams(
        pooling=ZymCTRLPoolingType.MEAN,
        layer=-1,  # Last hidden layer
    ),
    items=[
        ZymCTRLEncodeRequestItem(
            sequence="MKTVRQERLKSIVRILERSKEPVSGAQLAEELSVSRQVIVQDI",
            ec_number="3.5.5.1",  # Optional EC context
        ),
    ],
)

Confidence Metrics¶

Metric	Range	Interpretation
Perplexity	>0, no upper bound	< 1.5: excellent, 1.5-2.0: good, 2.0-4.0: moderate, >4.0: likely poor quality

Perplexity is computed on amino acid tokens only, excluding EC number and control tokens. This matches the paper's methodology and provides values directly comparable to the paper's quality thresholds. Lower perplexity indicates the generated sequence is more consistent with the model's learned distribution for that EC class.

Technical Glossary¶

EC number (Enzyme Commission number): Hierarchical four-level numerical classification of enzyme function (e.g., 4.2.1.1). Level 1 = reaction type, Level 2 = substrate type, Level 3 = specifics, Level 4 = individual enzyme.

Control tag: The EC number prepended to a sequence during training and inference, which conditions the model's generation on a specific catalytic function.

Perplexity: Exponential of the average cross-entropy loss. Measures how "surprised" the model is by a sequence. Lower values mean the sequence fits the model's learned enzyme distribution better.

Top-k sampling: At each generation step, only the top k most probable next tokens are considered. Lower k = more conservative; higher k = more diverse.

Repetition penalty: Multiplicative penalty applied to tokens that have already appeared in the generated sequence, discouraging repetitive patterns.

Architecture & training¶

Architecture¶

Model Type & Innovation¶

ZymCTRL is a conditional protein language model based on GPT-2 (decoder-only autoregressive transformer) that generates enzyme sequences conditioned on Enzyme Commission (EC) numbers. The key innovation is the use of EC numbers as control tags prepended to enzyme sequences during training, enabling zero-shot generation of enzymes for any catalytic function without fine-tuning.

Unlike general-purpose protein language models (e.g., ESM2, ProtGPT2) that generate unconditional protein sequences, ZymCTRL ties generation to a functional specification (the EC number). This is achieved through character-level tokenization of EC numbers (e.g., "2.7.1.1" becomes ["2", ".", "7", ".", "1", ".", "1"]), which enables the model to learn hierarchical relationships between enzyme classes and transfer knowledge across related EC categories.

The model builds on ProtGPT2 (Ferruz et al., 2022) by adding the EC conditioning mechanism. Where ProtGPT2 generates proteins from a learned general distribution, ZymCTRL generates enzymes from an EC-conditioned distribution.

Parameters & Layers¶

Component	Details
Architecture	GPT-2 Transformer (decoder-only, autoregressive)
Parameters	738M
Layers	36
Hidden dimensions	1280
Vocabulary	Character-level amino acid + EC digit tokens + special tokens
Positional encoding	Learned absolute positional embeddings
Maximum sequence length	1024 tokens (block size)
Special tokens	`<sep>`, `<start>`, `<end>`, `<\\|endoftext\\|>`, `<pad>`

Training Data¶

Property	Details
Dataset	UniProt enzyme sequences with EC annotations
Size	37 million enzyme sequences
Snapshot date	July 2022
Filtering	Sequences with validated EC number annotations from UniProt
EC coverage	All four levels of the EC hierarchy
Training format	`<ec_number><sep><start><amino_acid_sequence><end><\\|endoftext\\|>`

Known biases: - EC classes with more representatives in UniProt will have better generation quality - Broad-substrate EC classes (e.g., hexokinases EC 2.7.1.1) produce more heterogeneous outputs, reflecting the diversity in the training data - Under-represented or recently created EC classes may produce lower-quality sequences

Loss Function & Objective¶

Standard autoregressive language modeling: next-token prediction with cross-entropy loss across the full sequence (EC tokens + separator + amino acid tokens).

L = -sum over t of log P(x_t | x_{<t})

The model learns the joint distribution P(sequence | EC number) through teacher forcing on the concatenated EC + sequence format. At inference time, perplexity computed on the amino acid tokens only (excluding EC/control tokens) serves as a quality metric, with lower perplexity indicating sequences more consistent with the learned enzyme distribution.

Tokenization / Input Processing¶

Tokenizer type: Character-level (each amino acid and EC digit is a separate token)
Training format: <ec_number><sep><start><sequence><end><|endoftext|>
Special tokens:
<sep> -- separates EC number from sequence
<start> -- marks beginning of amino acid sequence
<end> -- marks end of amino acid sequence
<|endoftext|> -- standard GPT-2 end-of-text token
<pad> -- padding token (vocabulary ID 0)
Maximum length: 1024 tokens (includes EC number, separators, and sequence)
EC number encoding: Character-level (e.g., "3.5.5.1" tokenized as individual characters), enabling transfer learning across the EC hierarchy

Performance & Benchmarks¶

Published Benchmarks¶

Enzyme Generation Quality (Paper Figure 1b)¶

The paper evaluates generated sequences using perplexity as a proxy for quality, where lower perplexity indicates sequences more consistent with the natural enzyme distribution.

EC Class	Name	Avg Perplexity	Quality
2.7.1.2	Glucokinase	~1.1	Excellent
4.2.1.1	Carbonic anhydrase	Variable (best ~1.2)	Good
1.1.1.27	Lactate dehydrogenase	~2.3	Good

Key finding: Generated sequences average approximately 53% sequence identity to natural proteins, indicating the model produces genuinely novel sequences rather than memorizing training data.

Experimental Validation (Paper Figures 3-4)¶

Carbonic anhydrase (EC 4.2.1.1): Generated enzymes were experimentally validated, confirming catalytic activity
Lactate dehydrogenase (EC 1.1.1.27): Fine-tuning on this EC class improved generation quality

Verification Results¶

Test Case	Expected	Result	Status
Glucokinase (2.7.1.2) perplexity	LOW (<1.5)	1.11 avg, 1.06 min	PASS
LDH (1.1.1.27) perplexity	LOW	2.28 avg, 1.95 min	PASS
Carbonic anhydrase (4.2.1.1) perplexity	LOW	3.05 avg, 1.17 min	PASS
Implausible EC (9.9.9.9) perplexity	HIGH	8.92 avg, 8.46 min	PASS
Partial EC (1.1) perplexity	MEDIUM	5.15 avg, 2.80 min	PASS

Valid ECs produce average perplexity of 2.15 vs 8.92 for implausible ECs (4.1x difference), confirming the model differentiates between valid and random EC labels.

Comparison to Alternatives¶

Model	Task	Conditioning	When to Prefer
ZymCTRL	Enzyme generation	EC number	Need enzymes for specific catalytic function
ProtGPT2	General protein generation	None (unconditional)	Need general proteins, not necessarily enzymes
ESM2	Embeddings / scoring	None	Need variant effect prediction, not generation
ProGen2	Protein generation	Taxonomy + function tags	Need broader conditioning beyond EC numbers

Error Bars & Confidence¶

ZymCTRL generation is stochastic. Key variance characteristics:

Perplexity variance: Within an EC class, generated sequences show a range of perplexities. Well-represented classes (e.g., glucokinase) show tighter distributions; under-represented classes show wider variance.
Recommended workflow: Generate 100-1000 sequences, rank by perplexity, and keep the best ~5%. The model card recommends aiming for perplexity below ~1.5-1.75 (lower is better); the "best ~5%" is a separate selection heuristic rather than a fixed perplexity cutoff.
Seed control: The implementation supports explicit seeds for reproducibility. Without a seed, time-based entropy is used for diversity.

Strengths & Limitations¶

Pros¶

Zero-shot enzyme generation for any EC class without fine-tuning
EC number conditioning enables targeted catalytic function design
Character-level EC tokenization enables transfer learning across enzyme hierarchy
Perplexity provides a built-in quality metric for ranking generated sequences
Experimentally validated: generated carbonic anhydrases showed catalytic activity
Produces genuinely novel sequences (~53% identity to natural proteins)
Embedding extraction (encode action) enables downstream enzyme similarity analysis

Cons¶

Sequence-only model: no structural awareness or binding site specificity
No substrate specificity control within an EC class (e.g., cannot specify which sugar a kinase should phosphorylate)
Broad-substrate EC classes produce heterogeneous outputs
Maximum 1024 tokens limits generation of very long enzymes
Perplexity is a proxy for quality, not a direct measure of catalytic activity
Requires post-generation filtering and experimental validation

Known Failure Modes¶

Non-existent EC numbers (e.g., 9.9.9.9): Model generates sequences but with very high perplexity (>8), indicating poor quality. The model does not reject invalid EC numbers.
Partial EC numbers (e.g., "1.1" instead of "1.1.1.1"): Generates sequences but with higher uncertainty (perplexity ~5), since the model was trained predominantly on full 4-level EC numbers.
Under-represented EC classes: EC categories with few training examples produce lower-quality sequences with higher perplexity variance.
Repetition: Without sufficient repetition penalty, the model can fall into repetitive amino acid patterns.

Implementation Details¶

Inference Pipeline¶

Generate Action¶

Request (EC number + generation params)
  |-- 1. Validate EC number format (X.X.X.X or partial)
  |-- 2. Set random seeds (user-provided or time-based)
  |-- 3. Build prompt: "<ec_number><sep><start>"
  |-- 4. Tokenize prompt
  |-- 5. [GPU] Autoregressive generation (top-k sampling)
  |     |-- temperature, top_k, repetition_penalty control
  |     |-- Generate up to max_length tokens per sample
  |     |-- num_samples independent generations
  |-- 6. For each sample:
  |     |-- Calculate perplexity on amino acid tokens only
  |     |-- Decode and remove special tokens
  |-- 7. Sort by perplexity (ascending)
  |-- 8. Return sorted results

Encode Action¶

Request (sequence + optional EC number + pooling params)
  |-- 1. Validate amino acid sequence
  |-- 2. Format with training tokens:
  |     |-- With EC: "<ec><sep><start><sequence><end>"
  |     |-- Without EC: "<start><sequence><end>"
  |-- 3. Tokenize with padding and attention mask
  |-- 4. [GPU] Forward pass with output_hidden_states=True
  |-- 5. Extract hidden states from specified layer (default: last)
  |-- 6. Pool embeddings:
  |     |-- mean: average over non-padding positions
  |     |-- last: final non-padding token
  |     |-- per_residue: all non-padding token embeddings
  |-- 7. Return embeddings

Memory & Compute Profile¶

Configuration	GPU	Memory	Notes
Model weights	T4 (16GB)	~3 GB VRAM	738M params in FP32
Generation (1 sample, 256 tokens)	T4	~4 GB VRAM	Sequential autoregressive
Generation (20 samples, 1024 tokens)	T4	~6 GB VRAM	Max batch
Encoding (8 sequences, mean pooling)	T4	~5 GB VRAM	Max encode batch

Resource allocation: 2 CPU cores, 16 GB system RAM, T4 GPU, 10-minute timeout.

Determinism & Reproducibility¶

Setting	Value
Torch manual seed	Yes (user-provided or time-based)
CUDA manual seed	Yes (`torch.cuda.manual_seed_all`)
NumPy seed	Yes
Python random seed	Yes
cuDNN deterministic	Not explicitly set
cuDNN benchmark	Not explicitly set

Generate action: Stochastic by design (sampling-based). Reproducible with explicit seed parameter.
Encode action: Deterministic for the same input (no sampling involved).

Caching Behavior¶

Response caching is handled at the serving layer, not by the model container: - Cache key: Based on full request payload (EC number, params, sequence) - Generate action: Caching is less useful due to stochastic outputs (same EC + different seed = different results) - Encode action: Caching is effective since outputs are deterministic

Training Procedures¶

Training was performed by the original authors (AI4PD group) and is not reproducible here.

Hyperparameter	Value
Base model	GPT-2 architecture (trained from scratch)
Hardware	48 NVIDIA A100 GPUs
Training time	~15,000 GPU hours
Training data	37M enzyme sequences from UniProt (July 2022)
Block size	1024 tokens
Objective	Autoregressive language modeling (next-token prediction)

Versions & Changelog¶

Version	Date	Changes
v1	2024-12	Initial implementation with generate and encode actions

Biology¶

Molecule Coverage¶

Primary Molecule Type(s)¶

ZymCTRL is designed exclusively for enzymes -- proteins that catalyze biochemical reactions. The model covers enzymes from all domains of life (bacteria, archaea, eukaryota), as represented in UniProt's enzyme annotation database (July 2022 snapshot, 37M sequences).

Coverage characteristics: - Well-represented: Enzymes with abundant UniProt entries and well-established EC classifications (e.g., oxidoreductases, transferases, hydrolases) - Moderate coverage: Enzymes with fewer training examples or recently classified EC numbers - Limited coverage: Membrane-associated enzymes, multi-subunit enzyme complexes (model generates single chains), and enzymes requiring post-translational modifications for activity - Sequence length: Up to ~1000 amino acids (1024 token limit includes EC number and control tokens)

Cross-Applicability¶

Molecule Type	Applicability	Evidence	Caveats
Enzymes (primary)	High	37M training sequences, experimental validation	Best for EC classes with many representatives
Non-enzyme proteins	Not applicable	Not trained on non-enzyme proteins	Use ProtGPT2 or ESM2 for general proteins
Peptides	Not applicable	Too short for meaningful EC-conditioned generation	Minimum useful output ~50 residues
Antibodies	Not applicable	Not in training distribution	Use antibody-specific models
DNA/RNA	Not applicable	Protein-only model	Use nucleotide-specific models

Biological Problems Addressed¶

Problem 1: De Novo Enzyme Design¶

Biological context: Designing new enzymes with specific catalytic functions is a central challenge in protein engineering and synthetic biology. Traditional approaches rely on directed evolution (iterative rounds of random mutagenesis and screening) or rational design (structure-guided mutations). Both are expensive, slow, and limited to modifying existing enzymes.

How ZymCTRL addresses it: ZymCTRL generates entirely new enzyme sequences conditioned on an EC number, which specifies the desired catalytic reaction. The user provides an EC number (e.g., "4.2.1.1" for carbonic anhydrase), and the model generates novel amino acid sequences predicted to catalyze that reaction.

What the output means biologically: Each generated sequence is a candidate enzyme that, according to the model's learned distribution, is consistent with the structural and functional properties of enzymes in that EC class. The accompanying perplexity score indicates how well the sequence fits the learned distribution -- lower perplexity suggests higher likelihood of being a functional enzyme.

Practical workflow: 1. Specify the target EC number for the desired catalytic activity 2. Generate hundreds to thousands of candidate sequences 3. Rank by perplexity (aim for below ~1.5-1.75 for high confidence; lower is better) 4. Keep the best ~5% of candidates 5. Validate computationally (e.g., structure prediction with ESMFold/Chai-1, active site analysis) 6. Synthesize and test experimentally

Problem 2: Exploring Enzyme Sequence Space¶

Biological context: Natural enzymes represent a tiny fraction of possible functional sequences. Understanding the broader sequence space of functional enzymes is important for discovering novel catalytic mechanisms, improving enzyme stability, and finding sequences with properties not found in nature (e.g., thermostability, solvent tolerance).

How ZymCTRL addresses it: By generating sequences that are novel (average ~53% identity to natural proteins), ZymCTRL samples from regions of sequence space that evolution has not explored but that are predicted to be functional. This is fundamentally different from homology-based methods that are constrained to sequences similar to known proteins.

Biological significance: The 53% average identity to natural enzymes means generated sequences are in a "twilight zone" where they share fold-level similarity to natural enzymes but differ substantially in primary sequence, potentially yielding enzymes with new combinations of properties.

Problem 3: Enzyme Embedding and Functional Similarity Analysis¶

Biological context: Comparing enzymes by function rather than just sequence similarity is important for understanding enzyme evolution, predicting function of uncharacterized enzymes, and identifying functionally equivalent enzymes from different organisms.

How ZymCTRL addresses it: The encode action extracts embeddings from ZymCTRL's internal representations. Because the model was trained on EC-annotated sequences, its embeddings capture functional information beyond raw sequence similarity. Optionally providing an EC number as context further biases the embedding toward functional representation.

Applications: - Clustering enzymes by functional similarity rather than sequence similarity - Identifying functionally equivalent enzymes across organisms - Detecting enzymes with similar catalytic mechanisms but divergent sequences

Applied Use Cases¶

Use Case 1: Carbonic Anhydrase Design¶

Source: Munsamy et al. "Conditional language models enable the efficient design of proficient enzymes." bioRxiv (2024). DOI

The authors used ZymCTRL to generate novel carbonic anhydrases (EC 4.2.1.1), an enzyme class important in carbon capture, biosensors, and medical applications. Generated sequences were filtered by perplexity, structurally validated using ESMFold, and experimentally tested. Multiple designs showed catalytic activity, demonstrating that ZymCTRL can produce functional enzymes without fine-tuning.

Use Case 2: Lactate Dehydrogenase Fine-Tuning¶

Source: Munsamy et al. (same paper, Figure 4)

For more targeted design, the authors fine-tuned ZymCTRL on lactate dehydrogenase (EC 1.1.1.27) sequences. Fine-tuning improved generation quality for this specific EC class, demonstrating that the base model can be further specialized. This approach is relevant when users need many high-quality designs for a particular enzyme class and are willing to invest in fine-tuning.

Predecessor Models¶

ProtGPT2 (Ferruz et al., 2022): ZymCTRL's direct predecessor. Same GPT-2 architecture but trained on general UniRef50 proteins without EC conditioning. ProtGPT2 generates unconditional protein sequences; ZymCTRL adds the critical EC-conditioning mechanism for targeted enzyme design.

Complementary Models¶

ESM2 / ESMFold / Chai-1: Use for structural validation of ZymCTRL-generated sequences. Typical pipeline: ZymCTRL generates sequences, then structure prediction confirms proper folding and active site geometry.
ESM2 embeddings: Can be used alongside ZymCTRL embeddings for a complementary view -- ESM2 captures general protein properties while ZymCTRL embeddings are enzyme-function-aware.

Alternative Models¶

Alternative	Advantage over ZymCTRL	Disadvantage
ProtGPT2	Broader protein coverage (not limited to enzymes)	No functional conditioning; cannot target specific catalytic activity
ProGen2	Supports taxonomy and broader function tags	Not specifically optimized for enzyme EC hierarchy
ESM2 (zero-shot scoring)	Can score/rank existing sequences	Cannot generate new sequences de novo
Directed evolution (experimental)	Produces experimentally validated enzymes	Orders of magnitude slower and more expensive

Biological Background¶

Enzymes and the EC Classification System¶

Enzymes are biological catalysts -- proteins that accelerate chemical reactions in living organisms. Nearly all metabolic reactions in cells are catalyzed by enzymes, making them essential for life and valuable in biotechnology (industrial biocatalysis, pharmaceutical synthesis, diagnostics, bioremediation).

The Enzyme Commission (EC) number system is the standard hierarchical classification for enzyme function. It uses a four-level numerical code:

First level: General reaction type (1=oxidoreductases, 2=transferases, 3=hydrolases, 4=lyases, 5=isomerases, 6=ligases, 7=translocases)
Second level: Substrate type or bond acted upon
Third level: More specific substrate or co-factor
Fourth level: Specific enzyme

For example, EC 4.2.1.1 (carbonic anhydrase): class 4 (lyase), subclass 4.2 (carbon-oxygen lyases), sub-subclass 4.2.1 (hydro-lyases), enzyme 4.2.1.1 (carbonate dehydratase).

Why Computational Enzyme Design Matters¶

Engineering new enzymes is important for: - Industrial biocatalysis: Enzymes that function at high temperatures, extreme pH, or in organic solvents for chemical manufacturing - Pharmaceutical synthesis: Stereoselective catalysts for drug production - Environmental remediation: Enzymes that degrade pollutants or plastics - Carbon capture: Engineered carbonic anhydrases for CO2 sequestration - Diagnostics: Enzyme-based biosensors

Traditional enzyme engineering (directed evolution) requires multiple rounds of mutagenesis and screening, which is time-consuming and expensive. Computational design tools like ZymCTRL can dramatically accelerate this process by generating diverse candidate sequences for experimental testing, reducing the search space from astronomical (20^N possible sequences for an N-residue protein) to a manageable set of high-probability candidates.

Key Terminology¶

EC number: Enzyme Commission classification number specifying catalytic function (e.g., 3.5.5.1)
Perplexity: Information-theoretic measure of how well a sequence fits the model's learned distribution. Lower values indicate the model considers the sequence more "enzyme-like" for the given EC class. Values below ~1.5-1.75 are recommended for high-quality candidates (model card guidance).
Control tag: The EC number prepended to the sequence during training and generation, which conditions the model's output on a specific catalytic function
Zero-shot generation: Generating enzymes for an EC class without additional fine-tuning on that specific class
Biocatalysis: Using enzymes to catalyze chemical reactions in industrial processes

Sources & license¶

License: Apache-2.0 (text)

Papers

Conditional language models enable the efficient design of proficient enzymes — bioRxiv preprint, 2024 · DOI

Source repositories

huggingface: https://huggingface.co/AI4PD/ZymCTRL

Cite

@article{munsamy2024conditional,
  title={Conditional language models enable the efficient design of proficient enzymes},
  author={Munsamy, Geraldene and Illanes-Vicioso, Ramiro and Funcillo, Silvia and
          Nakou, Ioanna T. and Lindner, Sebastian and Ayres, Gavin and Sheehan, Lesley S.
          and Moss, Steven and Eckhard, Ulrich and Lorenz, Philipp and Ferruz, Noelia},
  journal={bioRxiv},
  year={2024},
  doi={10.1101/2024.05.03.592223}
}

ZymCTRL¶

At a glance¶

API & schema¶

generate¶

encode¶

Usage¶

Overview¶

Architecture¶

Capabilities & Limitations¶

Usage Examples¶

Confidence Metrics¶

Technical Glossary¶

Architecture & training¶

Architecture¶

Model Type & Innovation¶

Parameters & Layers¶

Training Data¶

Loss Function & Objective¶

Tokenization / Input Processing¶

Performance & Benchmarks¶

Published Benchmarks¶

Enzyme Generation Quality (Paper Figure 1b)¶

Experimental Validation (Paper Figures 3-4)¶

Verification Results¶

Comparison to Alternatives¶

Error Bars & Confidence¶

Strengths & Limitations¶

Pros¶

Cons¶

Known Failure Modes¶

Implementation Details¶

Inference Pipeline¶

Generate Action¶

Encode Action¶

Memory & Compute Profile¶

Determinism & Reproducibility¶

Caching Behavior¶

Training Procedures¶

Versions & Changelog¶

Biology¶

Molecule Coverage¶

Primary Molecule Type(s)¶

Cross-Applicability¶

Biological Problems Addressed¶

Problem 1: De Novo Enzyme Design¶

Problem 2: Exploring Enzyme Sequence Space¶

Problem 3: Enzyme Embedding and Functional Similarity Analysis¶

Applied Use Cases¶

Use Case 1: Carbonic Anhydrase Design¶

Use Case 2: Lactate Dehydrogenase Fine-Tuning¶

Related Models¶

Predecessor Models¶

Complementary Models¶

Alternative Models¶

Biological Background¶

Enzymes and the EC Classification System¶

Why Computational Enzyme Design Matters¶

Key Terminology¶

Sources & license¶

`generate`¶

`encode`¶