Omni-DNA¶

A family of multi-task DNA foundation models (20M--1B parameters) using BPE tokenization on the OLMo architecture, supporting embedding extraction and autoregressive log-probability scoring.

License: MIT · Molecules: dna · Tasks: embedding, property_prediction

Actions: encode, log_prob · Variants: 1

At a glance¶

Use it when

You need the largest-capacity BPE-tokenized DNA embedding model available in this catalog for complex downstream classification or regression tasks
You want true autoregressive log-probability scoring (not pseudo-log-likelihood) for DNA sequences using a BPE-tokenized model
You are benchmarking DNA foundation models and need a recent (2025) autoregressive architecture to compare against BERT-style (DNABERT-2) and byte-level (Evo/Evo2) approaches
You need a unified model for both embedding extraction and sequence fitness scoring and want the highest parameter count in a BPE-tokenized DNA model
You need a permissively licensed (MIT) large-capacity DNA model for commercial applications
You are developing multi-task genomic analysis pipelines and want a model explicitly designed for cross-modal versatility

Strengths

Unified autoregressive framework handles both embedding extraction and true log-probability scoring in a single architecture, unlike BERT-style models (DNABERT-2, NT) that provide only pseudo-log-likelihoods
1B-parameter model provides the largest capacity among BPE-tokenized DNA models in this catalog, potentially capturing more complex sequence patterns than DNABERT-2 (117M)
BPE tokenization learns data-driven subword units from genomic data, offering variable-length tokens that may capture biologically meaningful motifs as vocabulary entries
MIT license permits unrestricted commercial and academic use without licensing concerns
Most recent DNA foundation model (2025) in this catalog, incorporating latest training methodology and architectural design choices
Designed for cross-modal and multi-task learning, positioning it as a versatile backbone for diverse downstream genomic analyses
Dual-action API (encode + log_prob) covers both embedding extraction and sequence scoring

Limitations

Very recently published (2025) with no independent benchmark validation yet — performance claims rely solely on the original paper
No generative capability — despite autoregressive architecture, only scoring and embedding actions are exposed; cannot design or sample novel DNA sequences
No applied literature or downstream adoption studies available as of 2026-03, making real-world reliability and transferability unproven
Single deployed variant (1B) requires an L4 GPU, offering no lightweight option for resource-constrained deployments unlike DNABERT-2 (T4)
BPE token-level log-probabilities require careful normalization for cross-sequence comparisons since BPE produces variable numbers of tokens for different sequences
Training data composition and multi-species coverage are less documented than NT's explicit 850+ genome catalog
Does not support RNA sequences — input validation rejects non-ACGT characters

Reach for something else when

You need a well-validated model with extensive independent benchmarking and published downstream applications (use dnabert2, which has a multi-year track record)
You need DNA sequence generation capability (use evo or evo2, which support autoregressive generation)
You need a lightweight model for high-throughput screening on T4 GPUs or CPUs (use dnabert2 at 117M parameters)
You need very long-range genomic context exceeding standard transformer windows (use evo2 with up to 1M-token context)
You need codon optimization or deterministic DNA feature extraction (use dna_chisel instead)
You are working with RNA sequences (use an RNA-specific model)
You need reproducibility in a production pipeline and cannot tolerate using a model without established independent validation

Alternatives

Model	Better when	Worse when
dnabert2	Well-validated on the GUE benchmark (36 datasets) with multiple independent benchmarking studies; lightweight (117M on T4) and established since 2023	Smaller capacity (117M vs 1B parameters), shorter context window (~4-8 kbp), and masked LM architecture provides only pseudo-log-likelihood rather than true autoregressive scoring
evo2	Generation capability, multi-domain training (prokaryotes + eukaryotes + viruses), up to 1M-token context, and embedding extraction from a 7B-parameter model	Much higher compute requirements; Omni-DNA provides a more lightweight autoregressive option for scoring and embedding tasks
evo	Established publication (Science 2024), generation capability, and longer context (8 kbp) with 7B parameters	Prokaryotic training bias, no embedding endpoint, and higher compute cost; Omni-DNA provides embeddings and broader species coverage in a single model

API & schema¶

Variants

Variant	Endpoint slug	GPU	CPU	Memory
1b	`omni-dna-1b`	l4	4.0	16 GB

Call an action with POST /api/v1/{slug}/{action} — the request envelope is {"items": [...], "params": {...}} and a success returns {"results": [...]}. See the HTTP API page for the base URL, error shape, and full contract.

`encode`¶

Call it

curl -X POST http://127.0.0.1:8000/api/v1/omni-dna-1b/encode \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {
      "sequence": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE"
    }
  ]
}'

Request — OmniDNAEncodeRequest

Field	Type	Required	Constraints	Description
`params`	OmniDNAEncodeRequestParams	no		Optional parameters controlling this action (defaults are used when omitted).
`items`	list[OmniDNAEncodeRequestItem]	yes	items 1–2	Batch of inputs to process in a single request. Up to 2 sequences per request.

Nested types

OmniDNAEncodeIncludeOptions

Allowed values: mean, last

OmniDNAEncodeRequestItem

Field	Type	Required	Constraints	Description
`sequence`	string	yes	len 1–2048	A DNA sequence (A/C/G/T).

OmniDNAEncodeRequestParams

Field	Type	Required	Constraints	Description
`include`	list[OmniDNAEncodeIncludeOptions]	no	default `['mean']`	Optional outputs to compute and include in the response.

Raw JSON Schema

{
  "$defs": {
    "OmniDNAEncodeIncludeOptions": {
      "enum": [
        "mean",
        "last"
      ],
      "title": "OmniDNAEncodeIncludeOptions",
      "type": "string"
    },
    "OmniDNAEncodeRequestItem": {
      "additionalProperties": false,
      "properties": {
        "sequence": {
          "description": "A DNA sequence (A/C/G/T).",
          "maxLength": 2048,
          "minLength": 1,
          "title": "Sequence",
          "type": "string"
        }
      },
      "required": [
        "sequence"
      ],
      "title": "OmniDNAEncodeRequestItem",
      "type": "object"
    },
    "OmniDNAEncodeRequestParams": {
      "additionalProperties": false,
      "properties": {
        "include": {
          "description": "Optional outputs to compute and include in the response.",
          "items": {
            "$ref": "#/$defs/OmniDNAEncodeIncludeOptions"
          },
          "title": "Include",
          "type": "array",
          "default": [
            "mean"
          ]
        }
      },
      "title": "OmniDNAEncodeRequestParams",
      "type": "object"
    }
  },
  "additionalProperties": false,
  "properties": {
    "params": {
      "$ref": "#/$defs/OmniDNAEncodeRequestParams",
      "description": "Optional parameters controlling this action (defaults are used when omitted)."
    },
    "items": {
      "description": "Batch of inputs to process in a single request. Up to 2 sequences per request.",
      "items": {
        "$ref": "#/$defs/OmniDNAEncodeRequestItem"
      },
      "maxItems": 2,
      "minItems": 1,
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "OmniDNAEncodeRequest",
  "type": "object"
}

Response — OmniDNAEncodeResponse

Field	Type	Required	Constraints	Description
`results`	list[OmniDNAEncodeResponseResult]	yes		Per-input results, returned in the same order as the request items.

Nested types

OmniDNAEncodeResponseResult

Field	Type	Required	Constraints	Description
`mean`	list[number] \| null	no		Mean-pooled embedding vector over non-padded BPE tokens; present when "mean" is in params.include.
`last`	list[number] \| null	no		Last-token embedding vector from the final hidden layer; present when "last" is in params.include.

Raw JSON Schema

{
  "$defs": {
    "OmniDNAEncodeResponseResult": {
      "exclude_none": true,
      "exclude_unset": true,
      "properties": {
        "mean": {
          "anyOf": [
            {
              "items": {
                "type": "number"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Mean-pooled embedding vector over non-padded BPE tokens; present when \"mean\" is in params.include.",
          "title": "Mean"
        },
        "last": {
          "anyOf": [
            {
              "items": {
                "type": "number"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Last-token embedding vector from the final hidden layer; present when \"last\" is in params.include.",
          "title": "Last"
        }
      },
      "title": "OmniDNAEncodeResponseResult",
      "type": "object"
    }
  },
  "properties": {
    "results": {
      "description": "Per-input results, returned in the same order as the request items.",
      "items": {
        "$ref": "#/$defs/OmniDNAEncodeResponseResult"
      },
      "title": "Results",
      "type": "array"
    }
  },
  "required": [
    "results"
  ],
  "title": "OmniDNAEncodeResponse",
  "type": "object"
}

`log_prob`¶

Call it

curl -X POST http://127.0.0.1:8000/api/v1/omni-dna-1b/log_prob \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {
      "sequence": "MKTAYIAKQRQISFVKSHFSRQLEERLGLIE"
    }
  ]
}'

Request — OmniDNALogProbRequest

Field	Type	Required	Constraints	Description
`items`	list[OmniDNALogProbRequestItem]	yes	items 1–2	Batch of inputs to process in a single request. Up to 2 sequences per request.

Nested types

OmniDNALogProbRequestItem

Field	Type	Required	Constraints	Description
`sequence`	string	yes	len 1–2048	A DNA sequence (A/C/G/T).

Raw JSON Schema

{
  "$defs": {
    "OmniDNALogProbRequestItem": {
      "additionalProperties": false,
      "properties": {
        "sequence": {
          "description": "A DNA sequence (A/C/G/T).",
          "maxLength": 2048,
          "minLength": 1,
          "title": "Sequence",
          "type": "string"
        }
      },
      "required": [
        "sequence"
      ],
      "title": "OmniDNALogProbRequestItem",
      "type": "object"
    }
  },
  "additionalProperties": false,
  "properties": {
    "items": {
      "description": "Batch of inputs to process in a single request. Up to 2 sequences per request.",
      "items": {
        "$ref": "#/$defs/OmniDNALogProbRequestItem"
      },
      "maxItems": 2,
      "minItems": 1,
      "title": "Items",
      "type": "array"
    }
  },
  "required": [
    "items"
  ],
  "title": "OmniDNALogProbRequest",
  "type": "object"
}

Response — OmniDNALogProbResponse

Field	Type	Required	Constraints	Description
`results`	list[OmniDNALogProbResponseResult]	yes		Per-input results, returned in the same order as the request items.

Nested types

OmniDNALogProbResponseResult

Field	Type	Required	Constraints	Description
`log_prob`	number	yes		Log-likelihood of the sequence under the model.

Raw JSON Schema

{
  "$defs": {
    "OmniDNALogProbResponseResult": {
      "properties": {
        "log_prob": {
          "description": "Log-likelihood of the sequence under the model.",
          "title": "Log Prob",
          "type": "number"
        }
      },
      "required": [
        "log_prob"
      ],
      "title": "OmniDNALogProbResponseResult",
      "type": "object"
    }
  },
  "properties": {
    "results": {
      "description": "Per-input results, returned in the same order as the request items.",
      "items": {
        "$ref": "#/$defs/OmniDNALogProbResponseResult"
      },
      "title": "Results",
      "type": "array"
    }
  },
  "required": [
    "results"
  ],
  "title": "OmniDNALogProbResponse",
  "type": "object"
}

Usage¶

One-line summary: A family of multi-task DNA foundation models (20M--1B parameters) using BPE tokenization on the OLMo architecture, supporting embedding extraction and autoregressive log-probability scoring.

Overview¶

Omni-DNA is a family of multi-task, cross-modal genomic transformers that can handle DNA-based tasks in a single auto-regressive framework. Built on the OLMo (Open Language Model) architecture, Omni-DNA uses BPE (byte-pair encoding) tokenization to learn data-driven subword units from DNA sequences, providing an alternative to fixed k-mer (Nucleotide Transformer) or byte-level (Evo) tokenization approaches.

The model provides two actions: embedding extraction (mean or last-token pooling) and log-probability scoring for DNA sequences.

Reference: Omni-DNA on HuggingFace

Architecture¶

Property	Value
Architecture	OLMo Transformer (AutoModelForCausalLM)
Tokenization	BPE, vocabulary size 4096
Training objective	Autoregressive (causal language modeling)
Max sequence length	2,048 nucleotides (characters)
Input alphabet	A, C, G, T only
License	MIT

For detailed architecture information, see MODEL.md.

Capabilities & Limitations¶

CAN be used for: - Extracting DNA sequence embeddings (mean-pooled or last-token) for downstream ML tasks - Scoring DNA sequences via total autoregressive log-probability - Zero-shot variant effect assessment by comparing log-probabilities - Batch processing of up to 2 sequences per request

CANNOT be used for: - DNA sequence generation (no generate endpoint) - Sequences containing ambiguous bases (N, R, Y, etc.) -- only A, C, G, T accepted - RNA sequences (U is not accepted) - Protein sequences (use ESM2 or similar)

Other considerations: - BPE tokenization means sequence length in tokens does not directly map to nucleotide count - Input is capped at 2,048 nucleotides (characters); the BPE tokenizer truncates at 2,048 tokens, but that bound is never reached given the character cap - DNA sequences are tokenized using a BPE tokenizer that may split a sequence into sub-tokens (e.g., "A", "AA", "TG", etc.)

Usage Examples¶

# Encode -- get mean embeddings
from models.omni_dna.schema import (
    OmniDNAEncodeIncludeOptions,
    OmniDNAEncodeRequest,
    OmniDNAEncodeRequestItem,
    OmniDNAEncodeRequestParams,
)

encode_request = OmniDNAEncodeRequest(
    params=OmniDNAEncodeRequestParams(
        include=[OmniDNAEncodeIncludeOptions.MEAN, OmniDNAEncodeIncludeOptions.LAST],
    ),
    items=[OmniDNAEncodeRequestItem(sequence="ACGTACGTACGTACGT")],
)

# Score DNA sequences
from models.omni_dna.schema import (
    OmniDNALogProbRequest,
    OmniDNALogProbRequestItem,
)

logprob_request = OmniDNALogProbRequest(
    items=[OmniDNALogProbRequestItem(sequence="ATGATGATGATGATG")]
)

Notes¶

BPE Encoding & Context: DNA sequences are segmented into BPE tokens (e.g., "A", "C", "AA", "TG", etc.). Input sequences are accepted up to 2,048 nucleotides; longer sequences must be split before submission.
DNA-only Validation: Input sequences must contain only the unambiguous nucleotide characters: A, C, G, and T.
Resource Usage: The 1B variant requires a GPU such as L4.
Multi-Task Capabilities: For advanced tasks (e.g., adding classification heads), the model could be loaded via AutoModelForSequenceClassification with the same identifier.

Architecture & training¶

Architecture¶

Model Type & Innovation¶

Omni-DNA is a family of multi-task DNA foundation models based on the OLMo (Open Language Model) transformer architecture, adapted for genomic sequences. The key innovation is a unified auto-regressive framework that handles multiple DNA tasks (embedding extraction, sequence scoring) within a single model, using BPE (byte-pair encoding) tokenization rather than character-level or fixed k-mer tokenization.

The model uses the ai2-olmo (Allen AI OLMo) architecture with custom DNA-specific tokenization. This design choice enables the model to learn data-driven subword units from DNA sequences, potentially capturing biologically meaningful motifs as tokens.

Parameters & Layers¶

Variant	Parameters	Architecture	GPU
omni-dna-1b	~1B	OLMo Transformer (CausalLM)	L4

Additional variants defined but not currently deployed: 20M, 60M, 116M, 300M, 700M.

Property	Value
Architecture	OLMo Transformer (AutoModelForCausalLM)
Tokenization	BPE (byte-pair encoding), vocabulary size 4096
Max sequence length	2,048 nucleotides (characters); BPE tokenizer cap is 2,048 tokens but is never reached given the character limit
Positional encoding	As per OLMo architecture

Training Data¶

Property	Details
Dataset	DNA sequences (multi-species)
Preprocessing	BPE tokenization with vocabulary of 4096 tokens

Training data details are described in Li et al. 2025 (arXiv:2502.03499).

Loss Function & Objective¶

Standard autoregressive (causal language modeling) objective:

L = -sum_{t=1}^{T} log P(x_t | x_{<t})

The model predicts the next BPE token given all preceding tokens, learning the statistical structure of DNA at a subword level.

Tokenization / Input Processing¶

Property	Details
Tokenizer	BPE (AutoTokenizer from HuggingFace)
Vocabulary size	4,096 tokens
Token examples	"A", "AA", "TG", "ACGT", etc.
Input alphabet	A, C, G, T only (unambiguous DNA)
Max length	2,048 nucleotides input cap (BPE token limit unreachable)
Special tokens	BOS, PAD
Batch size	2 sequences per request

Note: Input is capped at 2,048 nucleotides. Due to BPE tokenization, the number of nucleotides per token varies, but the character cap ensures the BPE token limit is never reached.

Performance & Benchmarks¶

Published Benchmarks¶

See Li et al. "Omni-DNA: A Unified Genomic Foundation Model for Cross-Modal and Multi-Task Learning" (arXiv:2502.03499) for published benchmark results.

BioLM Verification Results¶

Action	Test Input	Tolerance	Status
`encode`	DNA sequence, mean pooling	rel_tol=1e-4	PASS
`log_prob`	DNA sequence	rel_tol=1e-4	PASS

Tests cover the 1B variant only.

Comparison to Alternatives¶

Model	Key Advantage	Key Disadvantage
Omni-DNA (this)	BPE tokenization; unified multi-task framework	Newer, less validated than established models
Evo 2	Multi-domain training; generation capability	Much larger; no BPE tokenization
Nucleotide Transformer	6-mer tokenization; extensive benchmarks	No generation; masked LM only
DNABERT-2	Established; BPE tokenization	Shorter context; masked LM only

Error Bars & Confidence¶

encode and log_prob are deterministic (seeds set to 42)
Small floating-point differences may occur across GPU architectures

Strengths & Limitations¶

Pros¶

BPE tokenization learns data-driven DNA subword units
Unified framework for multiple DNA tasks
Multiple model sizes (20M--1B) for speed/quality tradeoffs (only 1B currently deployed)
Based on well-tested OLMo architecture
Supports batched input (batch_size=2)

Cons¶

BPE tokenization means sequence length in tokens does not directly map to nucleotide count
No generation endpoint (encode and log_prob only)
Less validated than Evo or Nucleotide Transformer families
Only the 1B variant is currently deployed

Known Failure Modes¶

Very short sequences: Sequences that tokenize to very few BPE tokens provide limited context
Highly repetitive DNA: Repetitive sequences may be over-compressed by BPE, leading to degenerate representations
Non-DNA input: Only A, C, G, T accepted; ambiguity codes and RNA are rejected

Implementation Details¶

Inference Pipeline¶

Request
  |-- 1. Validate input (A/C/G/T only, length <= 2048 nucleotides)
  |-- 2. Route to action:
  |
  |-- [encode]
  |     |-- Tokenize with BPE (padding, truncation)
  |     |-- Forward pass with output_hidden_states=True
  |     |-- Extract final hidden states [B, L, D]
  |     |-- Compute mean or last pooling over non-padded tokens
  |     |-- Return embeddings
  |
  |-- [log_prob]
        |-- Tokenize without special tokens
        |-- Forward pass
        |-- log_softmax over vocabulary dimension
        |-- Gather log P(token_i) at each position
        |-- Mask padded positions
        |-- Sum per sequence
        |-- Return total log-prob

Memory & Compute Profile¶

Variant	GPU	Memory	CPU
omni-dna-1b	L4	16 GB	4 cores

Determinism & Reproducibility¶

Setting	Value
`torch.manual_seed`	42
`torch.cuda.manual_seed_all`	42
Model mode	eval() with torch.no_grad()

Results are reproducible on the same GPU architecture. GPU memory snapshot is enabled for fast cold starts.

Caching Behavior¶

Response caching is handled upstream of the model container; the container itself is stateless: - GPU memory snapshots enabled (enable_memory_snapshot=True, enable_gpu_snapshot=True) - Cache key derived from action name, input payload, and model variant

Versions & Changelog¶

Version	Date	Changes
v1	--	Initial implementation with encode and log_prob actions; 1B variant

Biology¶

Molecule Coverage¶

Primary Molecule Type(s)¶

Omni-DNA is designed for genomic DNA sequences composed of the four canonical bases (A, C, G, T). It uses BPE (byte-pair encoding) tokenization, which learns data-driven subword units from DNA sequences -- potentially capturing biologically meaningful motifs as individual tokens.

The model is applicable to: - Coding regions: Exons, open reading frames, gene bodies - Non-coding DNA: Intergenic regions, regulatory elements - Genomic sequences of varying complexity: The BPE tokenizer handles both repetitive and complex sequences

Cross-Applicability¶

Molecule Type	Applicability	Evidence	Caveats
Genomic DNA	High	Primary training domain	Performance varies by organism representation in training data
Coding DNA	High	Codon patterns captured by BPE tokenization
Regulatory DNA	Moderate	BPE may capture regulatory motifs	Less validated than NT or DNABERT on regulatory benchmarks
Synthetic DNA	Moderate	Can score via log-probability	Novel motifs may be out-of-distribution for BPE vocabulary
RNA sequences	Not supported	Input validation rejects non-ACGT characters	Use RNA-specific models
Protein sequences	Not applicable	DNA-only model	Use ESM2 or similar protein LMs

Biological Problems Addressed¶

Problem 1: DNA Sequence Representation Learning¶

Why this matters: Producing fixed-dimensional numerical representations of DNA sequences is essential for downstream ML tasks -- gene classification, regulatory element identification, variant effect prediction, and sequence clustering. Unlike k-mer-based tokenization (Nucleotide Transformer) or byte-level tokenization (Evo), BPE tokenization learns variable-length subword units from the data itself.

How Omni-DNA addresses it: The encode action extracts embeddings from the final hidden layer of the transformer. Users can request mean-pooled (averaging over all non-padded tokens) or last-token embeddings. These serve as feature vectors for any downstream supervised or unsupervised task.

Biological meaning: The BPE tokenizer may learn biologically meaningful tokens -- common dinucleotides, codons, or regulatory motifs -- as individual vocabulary entries. This could provide an intermediate granularity between single-nucleotide models (Evo) and fixed 6-mer models (NT).

Problem 2: DNA Sequence Fitness Scoring¶

Why this matters: Evaluating whether a DNA sequence is consistent with natural genomic patterns is important for variant interpretation, synthetic biology, and gene annotation.

How Omni-DNA addresses it: The log_prob action computes the total autoregressive log-probability of each sequence. The model processes sequences through its causal language model, computes log-softmax over the BPE vocabulary, and sums the log-probabilities of the actual tokens at each position.

Interpreting scores: - More negative values indicate sequences less consistent with the training distribution - Scores are summed over BPE tokens (not nucleotides), so comparison across sequences of different lengths requires normalization - Wild-type vs. mutant comparisons can identify potentially deleterious mutations

Applied Use Cases¶

Use Case 1: Multi-Task DNA Analysis (Published)¶

Source: Li et al. "Omni-DNA: A Unified Genomic Foundation Model for Cross-Modal and Multi-Task Learning." arXiv:2502.03499 (2025).

The paper demonstrates that a unified auto-regressive framework can handle multiple DNA tasks within a single model, potentially simplifying pipelines that currently require separate models for embedding extraction and sequence scoring.

Using Omni-DNA embeddings alongside protein embeddings (from ESM2) and deterministic DNA features (from DNA-Chisel) for multi-modal genomic analysis workflows.

Complementary Models¶

ESM2: Protein language model. For DNA-to-protein workflows, use Omni-DNA for DNA analysis and ESM2 for protein analysis.
Evo / Evo2: Autoregressive DNA models with generation capability. Use when sequence generation is needed.
DNA-Chisel: Deterministic feature extraction. Provides interpretable features that complement learned embeddings.

Alternative Models¶

Alternative	Advantage over Omni-DNA	Disadvantage vs Omni-DNA
Nucleotide Transformer	Established benchmarks; multi-species training	Fixed 6-mer tokenization; no autoregressive scoring
Evo 2	Generation capability; multi-domain training	Much larger model; byte-level only
DNABERT-2	Also uses BPE; well-benchmarked	Shorter context; masked LM only

Biological Background¶

DNA (deoxyribonucleic acid) encodes genetic information as a sequence of four nucleotide bases: adenine (A), cytosine (C), guanine (G), and thymine (T). Language models for DNA learn the statistical patterns in genomic sequences -- codon usage, regulatory motifs, compositional biases -- through self-supervised training objectives.

Key concepts relevant to Omni-DNA:

BPE tokenization: Byte-pair encoding is a subword tokenization algorithm that iteratively merges the most frequent pairs of tokens. Applied to DNA, BPE discovers variable-length "words" in the genomic sequence -- common dinucleotides, codons, or short motifs -- as vocabulary entries. This provides a data-driven alternative to fixed k-mer tokenization.
Autoregressive scoring: The model predicts each BPE token given all preceding tokens. The total log-probability of a sequence reflects how well it conforms to the patterns learned during training.
Embedding extraction: The hidden states of the transformer capture contextual information about each position in the sequence. Mean pooling over positions yields a fixed-dimensional representation of the entire sequence.
Multi-task learning: Training a single model on multiple objectives (or using a single architecture for multiple downstream tasks) can improve generalization by forcing the model to learn broadly useful representations.

Sources & license¶

License: MIT (text) — Upstream weights are distributed under MIT per the HuggingFace model card (license: mit).

Papers

Omni-DNA: A Unified Genomic Foundation Model for Cross-Modal and Multi-Task Learning — arXiv preprint, 2025 · arXiv

Source repositories

huggingface: https://huggingface.co/zehui127/Omni-DNA-1B

Omni-DNA¶

At a glance¶

API & schema¶

encode¶

log_prob¶

Usage¶

Overview¶

Architecture¶

Capabilities & Limitations¶

Usage Examples¶

Notes¶

Architecture & training¶

Architecture¶

Model Type & Innovation¶

Parameters & Layers¶

Training Data¶

Loss Function & Objective¶

Tokenization / Input Processing¶

Performance & Benchmarks¶

Published Benchmarks¶

BioLM Verification Results¶

Comparison to Alternatives¶

Error Bars & Confidence¶

Strengths & Limitations¶

Pros¶

Cons¶

Known Failure Modes¶

Implementation Details¶

Inference Pipeline¶

Memory & Compute Profile¶

Determinism & Reproducibility¶

Caching Behavior¶

Versions & Changelog¶

Biology¶

Molecule Coverage¶

Primary Molecule Type(s)¶

Cross-Applicability¶

Biological Problems Addressed¶

Problem 1: DNA Sequence Representation Learning¶

Problem 2: DNA Sequence Fitness Scoring¶

Applied Use Cases¶

Use Case 1: Multi-Task DNA Analysis (Published)¶

Use Case 2: Cross-Modal Genomic Feature Engineering (Anticipated)¶

Related Models¶

Complementary Models¶

Alternative Models¶

Biological Background¶

Sources & license¶

`encode`¶

`log_prob`¶