vllm.models.deepseek_v41.nvidia.ops.engram_pool
¶
DSv4.1 Engram tables in an Engram pool (vllm.distributed.engram_pool): pool
rows hold values then E8M0 scales; layers unpack MXFP8 WKV input as rows land.
Classes:
-
PoolEngram–An Engram layer whose table is split over an Engram pool.
-
PoolEngramBatch–Stage every layer's MXFP8 WKV input from one pool fetch per step: each
-
PoolEngramEmbedding–A table whose rows an Engram pool splits over GPUs of several engines;
PoolEngram
¶
Bases: Engram
An Engram layer whose table is split over an Engram pool.
Source code in vllm/models/deepseek_v41/nvidia/ops/engram_pool.py
PoolEngramBatch
¶
Bases: EngramBatch
Stage every layer's MXFP8 WKV input from one pool fetch per step: each
layer's rows are unpacked as they land (EngramPrefetch).
Source code in vllm/models/deepseek_v41/nvidia/ops/engram_pool.py
PoolEngramEmbedding
¶
Bases: ParallelEngramEmbedding
A table whose rows an Engram pool splits over GPUs of several engines; its parameters are views of this GPU's shard, so loaders fill it in place.
Methods:
-
lookup–Look up every head of
indicesinto BF16out. -
process_weights_after_loading–Serve once every GPU of the group has loaded its rows.
Source code in vllm/models/deepseek_v41/nvidia/ops/engram_pool.py
lookup(indices, out, background=False)
¶
Look up every head of indices into BF16 out.
Source code in vllm/models/deepseek_v41/nvidia/ops/engram_pool.py
process_weights_after_loading()
¶
Serve once every GPU of the group has loaded its rows.
Source code in vllm/models/deepseek_v41/nvidia/ops/engram_pool.py
_dequant_rows_kernel(packed, out, num_rows, ROW_STRIDE, DIM, PACKED_FP4, BLOCK)
¶
Dequantize pool rows (FP8 or MXFP4 values, then E8M0 scales) into BF16.
Source code in vllm/models/deepseek_v41/nvidia/ops/engram_pool.py
_unpack_fp8_kernel(rows, slots, ready, values, scales, num_tokens, table, HEADS, DIM, ROW_STRIDE, PLACED, BLOCK)
¶
Split table table's fetched [token][head] rows into MXFP8 values and
F8_128x4 scales; scales past num_tokens zero-pad the 128-row tile.