AbLang2¶

Germline-debiased paired antibody language model that produces sequence embeddings, residue embeddings, likelihood scores, and sequence restoration for paired heavy-light chain antibody sequences.

License: BSD-3-Clause · Molecules: antibody · Tasks: embedding, sequence_completion, property_prediction

Actions: encode, predict, generate, log_prob · Variants: 1

At a glance¶

Use it when

You have paired heavy and light chain sequences and need germline-debiased embeddings for clustering, property prediction, or repertoire analysis that should not be confounded by V-gene identity
You need to restore missing or ambiguous residues in antibody sequences from NGS or single-cell sequencing, especially when the paired chain provides useful context for CDR restoration
You want to score engineered or humanized antibody variants for sequence plausibility using log-probability, to filter out sequences that deviate too far from natural antibody patterns
You need per-position likelihood profiles to identify structurally or functionally constrained positions in an antibody sequence for rational mutagenesis planning
You are building a multi-model pipeline where SADIE provides numbering/annotation, AbLang2 provides embeddings and scoring, and AntiFold or AbodyBuilder3 provides structure-based design
You want a single antibody language model that covers embedding, generation, and scoring without needing to deploy separate models for each task

Strengths

Germline-debiased representations that focus on functional antibody properties rather than germline gene identity, making embeddings more informative for distinguishing functionally distinct antibodies within the same germline family
Four complementary actions (encode, predict, generate, log_prob) in a single model, covering embedding, per-position likelihood, sequence restoration, and global log-probability scoring
Paired heavy-light chain modeling captures inter-chain co-evolutionary signals critical for accurate paratope representation, unlike single-chain models that miss VH-VL interface effects
CPU-only inference (4 GB RAM, no GPU) enables cost-effective high-throughput screening of large antibody libraries without GPU infrastructure
Sequence restoration (generate action) fills missing residues using paired heavy-light context, which is particularly valuable for CDR recovery from noisy NGS or single-cell sequencing data
Trained on the Observed Antibody Space (OAS) paired dataset, one of the largest curated collections of paired antibody sequences, ensuring broad coverage of human and mouse germline families
Focal-loss training explicitly reduces germline bias, validated by published benchmarks showing AbLang2 embeddings separate antibodies by functional properties rather than V-gene usage
BSD-3-Clause license with no commercial restrictions, suitable for integration into proprietary therapeutic antibody development pipelines

Limitations

Requires both heavy and light chain sequences as input; cannot process single-chain, unpaired, or nanobody sequences -- use IgBERT (unpaired variant) for those cases
Custom Python library (not HuggingFace Transformers compatible), which limits interoperability with standard NLP/ML ecosystems and makes fine-tuning less straightforward
Processes variable domains only; constant regions (CH1, CL, Fc) are not modeled, so isotype-specific properties cannot be captured
CDR-H3 predictions may have higher uncertainty due to extreme sequence diversity in this region, particularly for unusually long CDR-H3 loops (>20 residues)
No structure awareness: predictions are purely sequence-based and cannot account for backbone conformation, which limits accuracy for tasks where 3D context matters
Training data is biased toward human and mouse antibodies; performance on antibodies from less-represented species (rabbit, chicken, camelid conventional antibodies) is less validated
Single variant only (no size options), so there is no speed/quality tradeoff available -- the model is either used at full size or not at all

Reach for something else when

You only have unpaired heavy or light chain sequences without the complementary chain -- use IgBERT (unpaired variant) which handles single-chain inputs natively
You are working with nanobody (VHH) or single-domain antibody sequences -- AbLang2 requires paired heavy and light chains and cannot process single-domain inputs; use AntiFold for structure-based VHH design or ImmuneFold for VHH structure prediction
You need 3D structure prediction of the antibody Fv region -- use AbodyBuilder3 or ImmuneFold for structure prediction from sequence
You need structure-conditioned sequence design (inverse folding) -- use AntiFold, which requires a backbone structure but produces structurally compatible sequences
You need to compare antibody sequences with non-antibody proteins -- use ESM-2, which covers the full protein universe
You need to predict specific developability properties (viscosity, thermostability) rather than general sequence plausibility -- use DeepViscosity for antibody viscosity or TemBERTure for thermostability (Tm)
You need T-cell receptor (TCR) analysis -- AbLang2 is not trained on TCR sequences; use ImmuneBuilder or ImmuneFold for TCR structure or SADIE for TCR annotation

Alternatives

Model	Better when	Worse when
igbert	IgBERT offers both paired and unpaired variants, handling single-chain antibody sequences that AbLang2 cannot process; it is HuggingFace-compatible for easier integration and fine-tuning	IgBERT lacks germline debiasing, so its embeddings may conflate germline gene identity with functional properties; AbLang2's focal-loss training produces more functionally informative representations
igt5	IgT5 uses a T5 architecture with relative position biases that may better capture long-range dependencies; it also offers both paired and unpaired variants	IgT5 is embedding-only (encode action) with no sequence generation, likelihood scoring, or restoration capabilities that AbLang2 provides
esm2	ESM-2 covers all proteins (not just antibodies), offers 5 size variants for speed/quality tradeoff, and has the largest downstream ecosystem of fine-tuned models	ESM-2 is not antibody-specialized, cannot model paired heavy-light chains, and lacks germline debiasing -- producing less informative representations for antibody-specific tasks

API & schema¶

Call an action with POST /api/v1/{slug}/{action} — the request envelope is {"items": [...], "params": {...}} and a success returns {"results": [...]}. See the HTTP API page for the base URL, error shape, and full contract.

`encode`¶

Call it

curl -X POST http://127.0.0.1:8000/api/v1/ablang2/encode \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {
      "heavy_chain": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE",
      "light_chain": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE"
    }
  ]
}'

Request — AbLang2EncodeRequest

Field	Type	Required	Constraints	Description
`params`	AbLang2EncodeParams	no		Optional parameters controlling this action (defaults are used when omitted).
`items`	list[AbLang2SequenceItem]	yes	items 1–32	Batch of inputs to process in a single request. Up to 32 sequences per request.

Nested types

AbLang2EncodeOptions

Allowed values: seqcoding, rescoding

AbLang2EncodeParams

Field	Type	Required	Constraints	Description
`include`	AbLang2EncodeOptions \| null	no	default `seqcoding`	Embedding mode; "seqcoding" returns a single pooled vector per pair, "rescoding" returns per-residue vectors.
`align`	boolean \| null	no	default `False`	Align residue embeddings to a standard antibody numbering scheme (rescoding only; not yet supported, must remain false).

AbLang2SequenceItem

Field	Type	Required	Constraints	Description
`heavy_chain`	string	yes	len 1–1024	Antibody heavy-chain amino-acid sequence.
`light_chain`	string	yes	len 1–1024	Antibody light-chain amino-acid sequence.

Raw JSON Schema

{
  "$defs": {
    "AbLang2EncodeOptions": {
      "enum": [
        "seqcoding",
        "rescoding"
      ],
      "title": "AbLang2EncodeOptions",
      "type": "string"
    },
    "AbLang2EncodeParams": {
      "additionalProperties": false,
      "properties": {
        "include": {
          "anyOf": [
            {
              "$ref": "#/$defs/AbLang2EncodeOptions"
            },
            {
              "type": "null"
            }
          ],
          "default": "seqcoding",
          "description": "Embedding mode; \"seqcoding\" returns a single pooled vector per pair, \"rescoding\" returns per-residue vectors."
        },
        "align": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": false,
          "description": "Align residue embeddings to a standard antibody numbering scheme (rescoding only; not yet supported, must remain false).",
          "title": "Align"
        }
      },
      "title": "AbLang2EncodeParams",
      "type": "object"
    },
    "AbLang2SequenceItem": {
      "additionalProperties": false,
      "description": "A single item in an AbLang2 request, which must have heavy + light sequences.",
      "properties": {
        "heavy_chain": {
          "description": "Antibody heavy-chain amino-acid sequence.",
          "maxLength": 1024,
          "minLength": 1,
          "title": "Heavy Chain",
          "type": "string"
        },
        "light_chain": {
          "description": "Antibody light-chain amino-acid sequence.",
          "maxLength": 1024,
          "minLength": 1,
          "title": "Light Chain",
          "type": "string"
        }
      },
      "required": [
        "heavy_chain",
        "light_chain"
      ],
      "title": "AbLang2SequenceItem",
      "type": "object"
    }
  },
  "additionalProperties": false,
  "properties": {
    "params": {
      "$ref": "#/$defs/AbLang2EncodeParams",
      "description": "Optional parameters controlling this action (defaults are used when omitted)."
    },
    "items": {
      "description": "Batch of inputs to process in a single request. Up to 32 sequences per request.",
      "items": {
        "$ref": "#/$defs/AbLang2SequenceItem"
      },
      "maxItems": 32,
      "minItems": 1,
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "AbLang2EncodeRequest",
  "type": "object"
}

Response — AbLang2EncodeResponse

Field	Type	Required	Constraints	Description
`results`	list[AbLang2EncodeResult]	yes		Per-input results, returned in the same order as the request items.
`number_alignment`	list[string] \| null	no		Per-residue antibody numbering labels, populated when alignment is requested (rescoding only); null otherwise.

Nested types

AbLang2EncodeResult

Field	Type	Required	Constraints	Description
`embeddings`	list[number] \| null	no		Germline-debiased sequence-level embedding vector for the antibody pair.
`residue_embeddings`	list[list[number]] \| null	no		Per-residue embedding vectors.

Raw JSON Schema

{
  "$defs": {
    "AbLang2EncodeResult": {
      "properties": {
        "embeddings": {
          "anyOf": [
            {
              "items": {
                "type": "number"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Germline-debiased sequence-level embedding vector for the antibody pair.",
          "title": "Embeddings"
        },
        "residue_embeddings": {
          "anyOf": [
            {
              "items": {
                "items": {
                  "type": "number"
                },
                "type": "array"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Per-residue embedding vectors.",
          "title": "Residue Embeddings"
        }
      },
      "title": "AbLang2EncodeResult",
      "type": "object"
    }
  },
  "properties": {
    "results": {
      "description": "Per-input results, returned in the same order as the request items.",
      "items": {
        "$ref": "#/$defs/AbLang2EncodeResult"
      },
      "title": "Results",
      "type": "array"
    },
    "number_alignment": {
      "anyOf": [
        {
          "items": {
            "type": "string"
          },
          "type": "array"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Per-residue antibody numbering labels, populated when alignment is requested (rescoding only); null otherwise.",
      "title": "Number Alignment"
    }
  },
  "required": [
    "results"
  ],
  "title": "AbLang2EncodeResponse",
  "type": "object"
}

`predict`¶

Call it

curl -X POST http://127.0.0.1:8000/api/v1/ablang2/predict \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {
      "heavy_chain": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE",
      "light_chain": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE"
    }
  ]
}'

Request — _AbLang2LikelihoodRequest

Field	Type	Required	Constraints	Description
`items`	list[AbLang2SequenceItem]	yes	items 1–32	Batch of inputs to process in a single request. Up to 32 sequences per request.

Nested types

AbLang2SequenceItem

Field	Type	Required	Constraints	Description
`heavy_chain`	string	yes	len 1–1024	Antibody heavy-chain amino-acid sequence.
`light_chain`	string	yes	len 1–1024	Antibody light-chain amino-acid sequence.

Raw JSON Schema

{
  "$defs": {
    "AbLang2SequenceItem": {
      "additionalProperties": false,
      "description": "A single item in an AbLang2 request, which must have heavy + light sequences.",
      "properties": {
        "heavy_chain": {
          "description": "Antibody heavy-chain amino-acid sequence.",
          "maxLength": 1024,
          "minLength": 1,
          "title": "Heavy Chain",
          "type": "string"
        },
        "light_chain": {
          "description": "Antibody light-chain amino-acid sequence.",
          "maxLength": 1024,
          "minLength": 1,
          "title": "Light Chain",
          "type": "string"
        }
      },
      "required": [
        "heavy_chain",
        "light_chain"
      ],
      "title": "AbLang2SequenceItem",
      "type": "object"
    }
  },
  "additionalProperties": false,
  "properties": {
    "items": {
      "description": "Batch of inputs to process in a single request. Up to 32 sequences per request.",
      "items": {
        "$ref": "#/$defs/AbLang2SequenceItem"
      },
      "maxItems": 32,
      "minItems": 1,
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "_AbLang2LikelihoodRequest",
  "type": "object"
}

Response — _AbLang2LikelihoodResponse

Field	Type	Required	Constraints	Description
`results`	list[AbLang2LikelihoodResult]	yes		Per-input results, returned in the same order as the request items.

Nested types

AbLang2LikelihoodResult

Field	Type	Required	Description
`logits`	list[list[number]]	yes	Per-position logits over the model vocabulary.
`sequence_tokens`	list[string]	yes	Per-position input tokens, aligned with the logits.
`vocab_tokens`	list[string]	yes	Vocabulary token order corresponding to the logits columns.

Raw JSON Schema

{
  "$defs": {
    "AbLang2LikelihoodResult": {
      "properties": {
        "logits": {
          "description": "Per-position logits over the model vocabulary.",
          "items": {
            "items": {
              "type": "number"
            },
            "type": "array"
          },
          "title": "Logits",
          "type": "array"
        },
        "sequence_tokens": {
          "description": "Per-position input tokens, aligned with the logits.",
          "items": {
            "type": "string"
          },
          "title": "Sequence Tokens",
          "type": "array"
        },
        "vocab_tokens": {
          "description": "Vocabulary token order corresponding to the logits columns.",
          "items": {
            "type": "string"
          },
          "title": "Vocab Tokens",
          "type": "array"
        }
      },
      "required": [
        "logits",
        "sequence_tokens",
        "vocab_tokens"
      ],
      "title": "AbLang2LikelihoodResult",
      "type": "object"
    }
  },
  "properties": {
    "results": {
      "description": "Per-input results, returned in the same order as the request items.",
      "items": {
        "$ref": "#/$defs/AbLang2LikelihoodResult"
      },
      "title": "Results",
      "type": "array"
    }
  },
  "required": [
    "results"
  ],
  "title": "_AbLang2LikelihoodResponse",
  "type": "object"
}

`generate`¶

Call it

curl -X POST http://127.0.0.1:8000/api/v1/ablang2/generate \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {
      "heavy_chain": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE",
      "light_chain": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE"
    }
  ]
}'

Request — _AbLang2RestoreRequest

Field	Type	Required	Constraints	Description
`params`	AbLang2RestoreParams	no		Optional parameters controlling this action (defaults are used when omitted).
`items`	list[AbLang2MissingSequenceItem]	yes	items 1–32	Batch of inputs to process in a single request. Up to 32 sequences per request.

Nested types

AbLang2MissingSequenceItem

Field	Type	Required	Constraints	Description
`heavy_chain`	string	yes	len 1–1024	Antibody heavy-chain amino-acid sequence; use "*" to mark positions for restoration.
`light_chain`	string	yes	len 1–1024	Antibody light-chain amino-acid sequence; use "*" to mark positions for restoration.

AbLang2RestoreParams

Field	Type	Required	Constraints	Description
`align`	boolean	no	default `False`	If true, return restored sequences using a standard antibody numbering scheme (not yet supported; must remain false).

Raw JSON Schema

{
  "$defs": {
    "AbLang2MissingSequenceItem": {
      "additionalProperties": false,
      "description": "For restore, we allow '*' placeholders inside the heavy/light sequences,\nwhich must still be valid length and contain at least 1 '*'.",
      "properties": {
        "heavy_chain": {
          "description": "Antibody heavy-chain amino-acid sequence; use \"*\" to mark positions for restoration.",
          "maxLength": 1024,
          "minLength": 1,
          "title": "Heavy Chain",
          "type": "string"
        },
        "light_chain": {
          "description": "Antibody light-chain amino-acid sequence; use \"*\" to mark positions for restoration.",
          "maxLength": 1024,
          "minLength": 1,
          "title": "Light Chain",
          "type": "string"
        }
      },
      "required": [
        "heavy_chain",
        "light_chain"
      ],
      "title": "AbLang2MissingSequenceItem",
      "type": "object"
    },
    "AbLang2RestoreParams": {
      "additionalProperties": false,
      "properties": {
        "align": {
          "default": false,
          "description": "If true, return restored sequences using a standard antibody numbering scheme (not yet supported; must remain false).",
          "title": "Align",
          "type": "boolean"
        }
      },
      "title": "AbLang2RestoreParams",
      "type": "object"
    }
  },
  "additionalProperties": false,
  "properties": {
    "params": {
      "$ref": "#/$defs/AbLang2RestoreParams",
      "description": "Optional parameters controlling this action (defaults are used when omitted)."
    },
    "items": {
      "description": "Batch of inputs to process in a single request. Up to 32 sequences per request.",
      "items": {
        "$ref": "#/$defs/AbLang2MissingSequenceItem"
      },
      "maxItems": 32,
      "minItems": 1,
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "_AbLang2RestoreRequest",
  "type": "object"
}

Response — AbLang2GenerateResponse

Field	Type	Required	Constraints	Description
`results`	list[AbLang2GenerateResponseResult]	yes		Per-input results, returned in the same order as the request items.

Nested types

AbLang2GenerateResponseResult

Field	Type	Required	Constraints	Description
`heavy_chain`	string	yes		Restored heavy-chain amino-acid sequence with all masked positions filled.
`light_chain`	string	yes		Restored light-chain amino-acid sequence with all masked positions filled.

Raw JSON Schema

{
  "$defs": {
    "AbLang2GenerateResponseResult": {
      "properties": {
        "heavy_chain": {
          "description": "Restored heavy-chain amino-acid sequence with all masked positions filled.",
          "title": "Heavy Chain",
          "type": "string"
        },
        "light_chain": {
          "description": "Restored light-chain amino-acid sequence with all masked positions filled.",
          "title": "Light Chain",
          "type": "string"
        }
      },
      "required": [
        "heavy_chain",
        "light_chain"
      ],
      "title": "AbLang2GenerateResponseResult",
      "type": "object"
    }
  },
  "properties": {
    "results": {
      "description": "Per-input results, returned in the same order as the request items.",
      "items": {
        "$ref": "#/$defs/AbLang2GenerateResponseResult"
      },
      "title": "Results",
      "type": "array"
    }
  },
  "required": [
    "results"
  ],
  "title": "AbLang2GenerateResponse",
  "type": "object"
}

`log_prob`¶

Call it

curl -X POST http://127.0.0.1:8000/api/v1/ablang2/log_prob \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {
      "heavy_chain": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE",
      "light_chain": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE"
    }
  ]
}'

Request — AbLang2LogProbRequest

Field	Type	Required	Constraints	Description
`items`	list[AbLang2SequenceItem]	yes	items 1–32	Batch of inputs to process in a single request. Up to 32 sequences per request.

Nested types

AbLang2SequenceItem

Field	Type	Required	Constraints	Description
`heavy_chain`	string	yes	len 1–1024	Antibody heavy-chain amino-acid sequence.
`light_chain`	string	yes	len 1–1024	Antibody light-chain amino-acid sequence.

Raw JSON Schema

{
  "$defs": {
    "AbLang2SequenceItem": {
      "additionalProperties": false,
      "description": "A single item in an AbLang2 request, which must have heavy + light sequences.",
      "properties": {
        "heavy_chain": {
          "description": "Antibody heavy-chain amino-acid sequence.",
          "maxLength": 1024,
          "minLength": 1,
          "title": "Heavy Chain",
          "type": "string"
        },
        "light_chain": {
          "description": "Antibody light-chain amino-acid sequence.",
          "maxLength": 1024,
          "minLength": 1,
          "title": "Light Chain",
          "type": "string"
        }
      },
      "required": [
        "heavy_chain",
        "light_chain"
      ],
      "title": "AbLang2SequenceItem",
      "type": "object"
    }
  },
  "additionalProperties": false,
  "properties": {
    "items": {
      "description": "Batch of inputs to process in a single request. Up to 32 sequences per request.",
      "items": {
        "$ref": "#/$defs/AbLang2SequenceItem"
      },
      "maxItems": 32,
      "minItems": 1,
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "AbLang2LogProbRequest",
  "type": "object"
}

Response — AbLang2LogProbResponse

Field	Type	Required	Constraints	Description
`results`	list[AbLang2LogProbResponseResult]	yes		Per-input results, returned in the same order as the request items.

Nested types

AbLang2LogProbResponseResult

Field	Type	Required	Constraints	Description
`log_prob`	number	yes		Pseudo-log-likelihood of the sequence under the model.

Raw JSON Schema

{
  "$defs": {
    "AbLang2LogProbResponseResult": {
      "properties": {
        "log_prob": {
          "description": "Pseudo-log-likelihood of the sequence under the model.",
          "title": "Log Prob",
          "type": "number"
        }
      },
      "required": [
        "log_prob"
      ],
      "title": "AbLang2LogProbResponseResult",
      "type": "object"
    }
  },
  "properties": {
    "results": {
      "description": "Per-input results, returned in the same order as the request items.",
      "items": {
        "$ref": "#/$defs/AbLang2LogProbResponseResult"
      },
      "title": "Results",
      "type": "array"
    }
  },
  "required": [
    "results"
  ],
  "title": "AbLang2LogProbResponse",
  "type": "object"
}

Usage¶

One-line summary: Germline-debiased paired antibody language model that produces sequence embeddings, residue embeddings, likelihood scores, and sequence restoration for paired heavy-light chain antibody sequences.

Overview¶

AbLang2 (Antibody Language Model 2) is a paired antibody language model developed by Olsen, Moal, and Deane at the Oxford Protein Informatics Group. It is trained with a masked language modeling objective on paired heavy-light chain sequences from the Observed Antibody Space (OAS) database, with explicit germline debiasing to produce representations that capture functional properties rather than germline gene identity.

AbLang2 improves on its predecessor AbLang by (1) modeling paired heavy-light sequences jointly to capture inter-chain dependencies and (2) correcting for germline bias, which otherwise dominates learned representations. This makes AbLang2 particularly suited for antibody engineering tasks where functional similarity matters more than germline origin.

Architecture¶

Property	Value
Architecture	Transformer encoder (BERT-style)
Training objective	Masked language modeling (MLM)
Training data	Observed Antibody Space (OAS) -- paired sequences
Max sequence length	1024 tokens per chain
Input format	Paired heavy+light: `<heavy>\\|<light>`
Positional encoding	Rotary position embeddings
License	BSD-3-Clause

Capabilities & Limitations¶

CAN be used for: - Generating germline-debiased sequence-level embeddings for paired antibodies (seqcoding) - Generating per-residue embeddings for paired antibodies (rescoding) - Computing per-position logits (unnormalized scores) for antibody sequences - Restoring missing residues in antibody sequences (sequence completion) - Zero-shot antibody sequence scoring via log-probability (log_prob)

CANNOT be used for: - Single-chain (unpaired) antibody analysis -- both heavy and light chains are required - Non-antibody proteins (use ESM-2 or similar) - Nanobody / VHH sequences (use the unpaired IgBERT or IgT5 variant) - 3D structure prediction (use AntiFold, ESMFold, or similar) - Antibody numbering and annotation (use SADIE)

Other considerations: - Runs on CPU only -- no GPU required - Batch size capped at 32 sequences per request - align=True option for rescoding/restore is not yet supported (requires ANARCI dependency)

Usage Examples¶

# Encode -- get sequence-level embeddings
from models.ablang2.schema import (
    AbLang2EncodeRequest,
    AbLang2EncodeParams,
    AbLang2SequenceItem,
)

encode_request = AbLang2EncodeRequest(
    params=AbLang2EncodeParams(include="seqcoding"),
    items=[
        AbLang2SequenceItem(
            heavy_chain="QVQLVQSGGQMKKPGSSVRVSCKASGYTFTNYGMNWVRQAPGQGLEWMGRI",
            light_chain="DIQMTQSPSSLSASVGDRVTITCKASQDVSTAVA",
        ),
    ],
)

# Generate -- restore missing residues
from models.ablang2.schema import (
    AbLang2GenerateRequest,
    AbLang2MissingSequenceItem,
    AbLang2RestoreParams,
)

generate_request = AbLang2GenerateRequest(
    params=AbLang2RestoreParams(align=False),
    items=[
        AbLang2MissingSequenceItem(
            heavy_chain="QVQLVQ*GGQMKKPGSSVRVSCKASGYTFTNYGMN**VRQAPGQGLEWMGRI",
            light_chain="DIQMTQSPSSLSA*VGDRVTITCKASQDVSTAVA",
        ),
    ],
)

# Predict log probability -- sequence scoring
from models.ablang2.schema import AbLang2LogProbRequest, AbLang2SequenceItem

log_prob_request = AbLang2LogProbRequest(
    items=[
        AbLang2SequenceItem(
            heavy_chain="QVQLVQSGGQMKKPGSSVRVSCKASGYTFTNYGMNWVRQAPGQGLEWMGRI",
            light_chain="DIQMTQSPSSLSASVGDRVTITCKASQDVSTAVA",
        ),
    ],
)

Architecture & training¶

Architecture¶

Model Type & Innovation¶

AbLang2 is a paired antibody language model based on the BERT (bidirectional transformer encoder) architecture. It is trained with a masked language modeling (MLM) objective on paired heavy-light chain antibody sequences from the Observed Antibody Space (OAS) database.

The key innovation of AbLang2 over its predecessor AbLang is the explicit handling of germline bias. The original AbLang model learned representations that were dominated by germline gene usage patterns, limiting its ability to capture functionally relevant somatic mutations. AbLang2 addresses this by training on paired heavy-light chain sequences and applying debiasing techniques that reduce the influence of germline identity, producing representations that better reflect antibody function, specificity, and developability rather than germline origin.

AbLang2 uses a transformer encoder with rotary position embeddings (via rotary-embedding-torch). It processes paired sequences in a concatenated format (<heavy>|<light>) with a separator token between chains, enabling the model to learn cross-chain dependencies.

Parameters & Layers¶

Property	Value
Architecture	Transformer encoder (BERT-style)
Training objective	Masked language modeling (MLM)
Positional encoding	Rotary position embeddings
Input format	Paired heavy+light: `<heavy>\\|<light>`
Max sequence length	1024 tokens per chain
Vocabulary	20 canonical amino acids + special tokens

AbLang-2 uses 12 transformer layers with an embedding size of 480 (12L + 480ES), based on the ESM-2 architecture with SwiGLU activation. It was pre-trained on 35.6M unpaired VH/VL sequences for 200,000 steps, then fine-tuned on 1.26M paired antibody sequences for 10,000 steps, using focal loss and modified masking (Olsen et al., 2024, Table 1).

Training Data¶

Property	Details
Dataset	Observed Antibody Space (OAS) -- paired sequences
Composition	Paired heavy-light chain antibody sequences from multiple species
Debiasing	Germline bias correction to reduce V-gene dominance

The training data consists of paired antibody sequences from OAS, which aggregates antibody repertoire sequencing data from published studies. The debiasing procedure reduces over-representation of common germline genes, ensuring the model learns sequence-function relationships beyond germline identity.

Loss Function & Objective¶

Masked language modeling (MLM) with cross-entropy loss:

L = -Sum_i log P(x_masked_i | x_visible)

During training, a fraction of input tokens are randomly masked and the model predicts the original amino acid at each masked position from the surrounding context, including cross-chain context from the paired heavy-light sequence.

Tokenization / Input Processing¶

Property	Details
Tokenizer	Character-level (one token per amino acid)
Input format	`<heavy>\\|<light>` with separator
Special tokens	Start/end tokens, separator `\\|`, `*` for masking
Max sequence per chain	1024 residues
Batch size	32 sequences

Input sequences are validated against the extended amino acid alphabet. For the generate action, * characters mark positions to be restored.

Performance & Benchmarks¶

Published Benchmarks¶

The AbLang2 paper demonstrates improvements over AbLang and general protein language models on antibody-specific tasks.

From Olsen et al. (2024), Tables 2-3: AbLang-2 achieves near-perfect germline residue prediction (perplexity ~1.1) while substantially improving non-germline (NGL) residue prediction compared to prior models. NGL perplexity scores for AbLang-2: VH FWR 9.92, VH CDR1/2 11.13, VH CDR3 12.47, VL FWR 10.09, VL CDR1/2 9.54, VL CDR3 10.77. By comparison, prior antibody LMs (AntiBERTy, AbLang-1) had NGL perplexities near or worse than random (20). The largest improvement came from switching to focal loss, which heavily skews training toward poorly predicted NGL residues. On clonotype mutation analysis, AbLang-2 achieved ~15% cumulative probability for known NGL residues in VH (vs. <2% for AntiBERTy and AbLang-1), while maintaining ~90-100% cumulative probability when germline residues are included.

Key findings from the paper: - Germline debiasing significantly improves representation quality for functional prediction tasks - Paired sequence modeling captures heavy-light chain co-evolution - AbLang2 outperforms general protein language models (ESM-2, ProtTrans) on antibody-specific tasks

BioLM Verification Results¶

The BioLM implementation uses official pre-trained weights loaded via ablang2.pretrained.pretrained(). Numerical verification is performed against golden reference outputs:

Metric	Threshold	Status
Relative tolerance	1e-4	PASS
Cosine distance	< 0.02	PASS

Tests cover encode (seqcoding + rescoding), predict, generate, and log_prob actions.

Comparison to Alternatives¶

Model	Type	Key Advantage	Key Disadvantage
AbLang2 (this)	Antibody LM	Germline-debiased, paired H+L	Antibody-specific only
IgBERT	Antibody LM	HuggingFace Transformers compatible	No germline debiasing
ESM-2	General protein LM	Broad protein coverage	Not antibody-specialized
AntiBERTy	Antibody LM	Unpaired chain modeling	No paired chain support

Error Bars & Confidence¶

AbLang2 is deterministic when seeds are set (as in the BioLM implementation). The same input produces the same output on the same hardware.

Sources of variability across runs: - Different GPU architectures may produce slightly different floating-point results (within 1e-4 relative tolerance)

Strengths & Limitations¶

Pros¶

Germline-debiased representations capture functional properties beyond V-gene identity
Paired heavy-light chain modeling captures inter-chain dependencies
Multiple output modes: seqcoding (sequence-level), rescoding (residue-level), likelihood, restore
CPU inference supported (no GPU required)
BSD-3-Clause licensed

Cons¶

Antibody-specific -- cannot be used for general proteins
Single variant only (no size options)
Requires both heavy and light chains (no single-chain mode)
align=True option for rescoding/restore not yet supported in BioLM (requires ANARCI)

Known Failure Modes¶

Non-antibody sequences: Model expects antibody sequences; non-antibody input will produce meaningless embeddings
Single-chain input: Both heavy and light chains are required; single-chain analysis is not supported
Very short CDRs: Sequences with atypically short CDR regions may produce lower-quality predictions
Non-standard numbering: The model does not enforce numbering schemes; sequences should be provided as raw amino acid strings

Implementation Details¶

Inference Pipeline¶

Request
  |-- 1. Validate sequences (alphabet, length, batch size)
  |-- 2. Format as paired input: (heavy, light) tuples
  |-- 3. Forward pass via ablang2 library
  |     |-- encode: seqcoding or rescoding mode
  |     |-- predict: likelihood mode (returns raw logits)
  |     |-- generate: restore mode (fill masked positions)
  |     |-- log_prob: logits -> log_softmax -> sum
  |-- 4. Post-process outputs to response schema
  |-- 5. Return typed response

Memory & Compute Profile¶

Property	Value
GPU	None (CPU only)
Memory	4 GB
CPU	2 cores
Batch size	32

The model runs entirely on CPU, making it cost-effective for high-throughput antibody screening.

Determinism & Reproducibility¶

Setting	Value
`torch.manual_seed`	42
`torch.cuda.manual_seed_all`	42 (if GPU available)
`torch.no_grad`	Yes (inference)
`model.freeze()`	Yes

Caching Behavior¶

Response caching is handled outside the model container. The model itself is stateless with respect to caching; the same request always produces the same output given the same weights.

Versions & Changelog¶

Version	Date	Changes
v1	2025-02-05	Initial implementation with encode, predict, generate, log_prob actions

Biology¶

Molecule Coverage¶

Primary Molecule Type(s)¶

AbLang2 is trained exclusively on paired antibody sequences -- heavy and light chain pairs from the Observed Antibody Space (OAS) database. The model operates on the amino acid level and processes both chains jointly, enabling it to capture inter-chain co-evolutionary signals.

AbLang2 handles different antibody regions with varying quality:

Framework regions (FWRs): Excellent representation quality. Framework residues are highly conserved and well-represented in training data.
Complementarity-determining regions (CDRs): Good representation quality. CDR diversity is well-captured, though CDR-H3 (the most variable region) may have higher prediction uncertainty.
Constant regions: Not applicable -- AbLang2 is designed for variable domain sequences only.
Nanobodies / VHH: Not supported -- requires paired heavy+light chains. For single-domain antibodies, use the unpaired IgBERT or IgT5 variant.

Cross-Applicability¶

Molecule Type	Applicability	Evidence	Caveats
IgG antibodies	High	Primary training molecule; paired H+L modeling	Both chains required
IgM / IgA	Moderate	Variable domains share structural features with IgG	Trained primarily on IgG; isotype-specific features not modeled
Nanobodies (VHH)	Not applicable	Requires paired sequences	Use the unpaired IgBERT or IgT5 variant instead
TCRs	Low	Some structural homology to antibodies	Not trained on TCR sequences; use TCR-specific models
General proteins	Not applicable	Antibody-specialized model	Use ESM-2 or similar general protein LMs

Biological Problems Addressed¶

Antibody Sequence Embedding¶

Problem: Antibody engineering requires numerical representations of antibody sequences for computational screening, clustering, and property prediction. Traditional approaches use germline gene identity, CDR sequence similarity, or hand-crafted features, which either conflate germline identity with functional properties or fail to capture the complex interplay between heavy and light chains.

How AbLang2 helps: The encode action with seqcoding mode produces a fixed-length embedding vector for paired heavy-light sequences. The rescoding mode produces per-residue embeddings. Both modes use germline-debiased representations that focus on functional properties rather than germline origin.

Biological meaning: A seqcoding embedding captures the "functional fingerprint" of an antibody pair, encoding information about binding specificity, developability, and structural stability. Antibodies targeting similar epitopes will have similar embeddings, even if they derive from different germline genes. Rescoding embeddings capture position-specific properties useful for identifying key contact residues and predicting the structural role of individual positions.

Antibody Sequence Restoration¶

Problem: Antibody sequences obtained from next-generation sequencing (NGS) or single-cell sequencing often contain missing or ambiguous residues due to sequencing errors, low coverage, or primer mismatches. Reconstructing these missing positions is essential for downstream analysis and expression.

How AbLang2 helps: The generate action takes sequences with * placeholder characters marking unknown positions and predicts the most likely amino acid at each position, using the full paired heavy-light context. This leverages the model's understanding of antibody sequence constraints to fill gaps with evolutionarily and structurally plausible residues.

Biological meaning: The restored residues reflect what the model considers most compatible with the surrounding sequence context, accounting for both intra-chain and inter-chain constraints. This is particularly useful for CDR restoration where the heavy and light chain context provides complementary information.

Antibody Likelihood Scoring¶

Problem: Evaluating whether an engineered or mutant antibody sequence is biophysically plausible requires assessing how well it fits known antibody sequence patterns. This is important for filtering designed sequences, assessing humanization quality, and predicting developability.

How AbLang2 helps: The predict action returns per-position likelihood scores, and log_prob returns total sequence log-probability. These scores quantify how "antibody-like" a sequence is under the model.

Biological meaning: A high likelihood score indicates the sequence is consistent with natural antibody patterns. Positions with low likelihood may indicate destabilizing mutations, unusual CDR conformations, or deviations from canonical structures. The germline debiasing ensures these scores reflect functional plausibility rather than proximity to common germline genes.

Applied Use Cases¶

AbLang2 embeddings and predictions are applicable to several antibody engineering workflows:

Antibody humanization assessment: Score humanized sequences for how well they match human antibody patterns (published)
Developability prediction: Use embeddings as features for predicting expression level, aggregation propensity, and stability (published)
CDR restoration from NGS data: Fill in missing residues from low-quality sequencing reads (published)
Antibody library design: Score designed sequences for biophysical plausibility (anticipated)
Repertoire analysis: Cluster and compare antibody repertoires using germline-debiased embeddings (anticipated)

Predecessor Models¶

AbLang (Olsen et al., 2022): The direct predecessor. A single-chain antibody language model trained on unpaired sequences from OAS. AbLang2 improves on AbLang by modeling paired sequences and debiasing germline influence.

Complementary Models¶

AbLang2 can be combined with other models in this catalog:

SADIE: Use SADIE for numbering and annotation, then AbLang2 for embedding and likelihood scoring
IgBERT/IgT5: Alternative antibody language models for comparison or ensemble approaches
AntiFold: Use AbLang2 embeddings for sequence selection, then AntiFold for structure-based inverse folding

Typical multi-model workflows: 1. Use SADIE predict to annotate and number antibody sequences 2. Use AbLang2 encode to generate embeddings for clustering and property prediction 3. Use AbLang2 log_prob to score engineered variants

Alternative Models¶

Alternative	Advantage Over AbLang2	Disadvantage vs AbLang2
IgBERT	HuggingFace-compatible, paired and unpaired variants	No germline debiasing
ESM-2	Broader protein coverage, more size variants	Not antibody-specialized, no paired chain modeling
AntiBERTy	Pre-trained on large OAS dataset	Single-chain only, no paired modeling

When to choose AbLang2: Use AbLang2 when you need germline-debiased paired antibody representations, sequence restoration, or likelihood scoring and always have both heavy and light chain sequences available.

When to choose alternatives: Consider IgBERT for unpaired (single-chain) analysis; consider ESM-2 when comparing antibodies to non-antibody proteins; consider the unpaired IgBERT or IgT5 variant for nanobody/VHH sequences.

Biological Background¶

Antibodies (immunoglobulins) are Y-shaped proteins produced by B cells of the adaptive immune system. Each antibody molecule consists of two identical heavy chains and two identical light chains, linked by disulfide bonds. The variable domains of the heavy chain (VH) and light chain (VL) together form the antigen-binding site (paratope) that determines binding specificity.

Germline genes and somatic hypermutation: Antibody variable domains are encoded by rearranged germline gene segments (V, D, J for heavy chains; V, J for light chains). After initial rearrangement, B cells undergo somatic hypermutation (SHM) in germinal centers, introducing point mutations that are selected for improved antigen binding (affinity maturation). Traditional antibody analysis methods heavily rely on germline gene assignment, which can bias functional comparisons -- two antibodies from different germline families may have converged on similar binding properties through SHM.

Germline bias in language models: When trained naively on antibody repertoire data, language models tend to learn germline gene usage patterns as the dominant signal, since germline identity explains most of the sequence variance in a repertoire. AbLang2 specifically addresses this by debiasing the training objective, forcing the model to learn signals beyond germline identity -- such as CDR conformation, paratope chemistry, and developability determinants.

Key terminology: - Heavy chain / Light chain: The two polypeptide chains forming an antibody; heavy chains are ~450 residues (variable + constant), light chains are ~214 residues. - Variable domain (VH/VL): The N-terminal domain of each chain containing the antigen-binding CDRs. - CDR (Complementarity-Determining Region): Three hypervariable loops per chain (CDR1, CDR2, CDR3) that directly contact the antigen. - Framework region (FWR): Conserved beta-sheet scaffold supporting the CDR loops. - Germline gene: The inherited, un-mutated gene segment encoding the initial antibody variable domain. - Somatic hypermutation (SHM): Process of introducing point mutations during B cell maturation. - OAS (Observed Antibody Space): Database of antibody repertoire sequencing data from published studies.

Sources & license¶

License: BSD-3-Clause (text)

Papers

AbLang2: Addressing the Antibody Germline Bias and Its Effect on Language Models for Improved Antibody Design — bioRxiv preprint, 2024 · DOI
AbLang: An antibody language model for completing antibody sequences — Bioinformatics Advances, 2022 · DOI

Source repositories

github: https://github.com/oxpig/AbLang2
pypi: https://pypi.org/project/ablang2/

Cite

@article{olsen2024ablang2,
  title={AbLang2: Addressing the Antibody Germline Bias and Its Effect on Language Models for Improved Antibody Design},
  author={Olsen, Tobias H and Moal, Iain H and Deane, Charlotte M},
  journal={bioRxiv},
  year={2024},
  doi={10.1101/2024.02.02.578678}
}

AbLang2¶

At a glance¶

API & schema¶

encode¶

predict¶

generate¶

log_prob¶

Usage¶

Overview¶

Architecture¶

Capabilities & Limitations¶

Usage Examples¶

Architecture & training¶

Architecture¶

Model Type & Innovation¶

Parameters & Layers¶

Training Data¶

Loss Function & Objective¶

Tokenization / Input Processing¶

Performance & Benchmarks¶

Published Benchmarks¶

BioLM Verification Results¶

Comparison to Alternatives¶

Error Bars & Confidence¶

Strengths & Limitations¶

Pros¶

Cons¶

Known Failure Modes¶

Implementation Details¶

Inference Pipeline¶

Memory & Compute Profile¶

Determinism & Reproducibility¶

Caching Behavior¶

Versions & Changelog¶

Biology¶

Molecule Coverage¶

Primary Molecule Type(s)¶

Cross-Applicability¶

Biological Problems Addressed¶

Antibody Sequence Embedding¶

Antibody Sequence Restoration¶

Antibody Likelihood Scoring¶

Applied Use Cases¶

Related Models¶

Predecessor Models¶

Complementary Models¶

Alternative Models¶

Biological Background¶

Sources & license¶

`encode`¶

`predict`¶

`generate`¶

`log_prob`¶