vllm.model_executor.kernels.linear.nvfp4
¶
Modules:
Classes:
-
NvFp4LinearKernel–Base class for NVFP4 quantized linear kernels.
-
NvFp4LinearLayerConfig–Configuration for an NVFP4 linear layer.
NvFp4LinearKernel
¶
Bases: LinearKernel[NvFp4LinearLayerConfig]
Base class for NVFP4 quantized linear kernels.
Each subclass implements a specific GEMM backend (CUTLASS, Marlin, etc).
The kernel selection mechanism iterates over registered subclasses in
priority order,calling is_supported and can_implement to find the best
match for the current hardware.
Methods:
-
prepare_input–Return 2D packed NVFP4 data with swizzled 128x4 block scales.
Source code in vllm/model_executor/kernels/linear/nvfp4/base.py
prepare_input(layer, x)
¶
Return 2D packed NVFP4 data with swizzled 128x4 block scales.
K is zero-padded to match the padded weight. Prequantized inputs are validated and padded without modifying the caller's activation.
Source code in vllm/model_executor/kernels/linear/nvfp4/base.py
NvFp4LinearLayerConfig
dataclass
¶
Configuration for an NVFP4 linear layer.
All NVFP4 layers share the same structure: packed uint8 weights (2 FP4 values per
byte), FP8-E4M3 per-block weight scales (group size 16), and scalar global
scales for both weights and activations. act_dtype is the activation
dtype of weight-only layers, when the caller knows it.