vllm.models.deepseek_v41.nvidia.ops.engram_fabric_table
¶
Engram tables in cuMem, shared between processes by FABRIC handle.
Host tables: one copy in host-NUMA memory per node-local DP group, which GPUs map with 2M pages instead of /dev/shm's 64K pages, so row lookups miss the TLB less.
Classes:
-
ExportedHostTable–What another rank needs to map a host table.
Functions:
-
fabric_host_memory_supported–Whether host-NUMA cuMem allocations can export FABRIC handles here.
-
share_host_table–Map one table on every rank of a node-local group, or None on all of them.
ExportedHostTable
¶
Bases: NamedTuple
What another rank needs to map a host table.
Source code in vllm/models/deepseek_v41/nvidia/ops/engram_fabric_table.py
_call_driver(fn, *args)
¶
Call a CUDA driver API and return its result, raising on failure.
Source code in vllm/models/deepseek_v41/nvidia/ops/engram_fabric_table.py
_create_table(num_bytes)
¶
Allocate and export a host table on this GPU's NUMA node.
Stops once the node would drop below its reserve. The kernel compacts to find 2M pages and fails the chunk, rather than OOM-killing, if it cannot.
Source code in vllm/models/deepseek_v41/nvidia/ops/engram_fabric_table.py
_gpu_view(address, num_bytes, owner=None)
¶
A uint8 tensor over mapped memory; torch retains owner with it.
Source code in vllm/models/deepseek_v41/nvidia/ops/engram_fabric_table.py
_map_table(table, num_bytes, handles)
¶
Map a host table, importing its handles unless given, and release them.
Returns CPU and GPU uint8 views; either keeps the mapping alive.
Source code in vllm/models/deepseek_v41/nvidia/ops/engram_fabric_table.py
fabric_host_memory_supported()
cached
¶
Whether host-NUMA cuMem allocations can export FABRIC handles here.
Source code in vllm/models/deepseek_v41/nvidia/ops/engram_fabric_table.py
share_host_table(num_bytes, group)
¶
Map one table on every rank of a node-local group, or None on all of them.
Rank 0 creates the table and broadcasts its handles; the others import them. Returns CPU and GPU uint8 views of the table.