vllm.model_executor.kernels.linear.mxfp6.base
¶
Classes:
-
MxFp6LinearKernel–Base class for MXFP6 quantized linear kernels.
-
MxFp6LinearLayerConfig–Configuration for an MXFP6 linear layer.
MxFp6LinearKernel
¶
Bases: LinearKernel[MxFp6LinearLayerConfig]
Base class for MXFP6 quantized linear kernels.
Each subclass implements a specific GEMM backend (CUTLASS, Marlin, etc).
The kernel selection mechanism iterates over registered subclasses in
priority order,calling is_supported and can_implement to find the best
match for the current hardware.
Source code in vllm/model_executor/kernels/linear/mxfp6/base.py
MxFp6LinearLayerConfig
dataclass
¶
Configuration for an MXFP6 linear layer.
All MXFP6 layers share the same structure: packed uint8 weights (4 FP6 values per 3 bytes) and per-block weight scales (group size 32).
Attributes:
-
weight_quant_key(QuantKey) –Identifies the weight quantization format. Can be kMxfp6E2M3Static or kMxfp6E3M2Static.
-
activation_quant_key(QuantKey | None) –Identifies the activation quantization format, or
Nonewhen activations must not be quantized.