vllm.model_executor.model_loader.reload.plan
¶
Load plan: which part of each checkpoint tensor this rank needs, and where in live memory those bytes go.
plan_loads traces the model's load_weights once (bake.py) on
the real tensors, between start_reload and finish_reload so that it
sees the layout direct mode actually restored (the NCCL engine does it at its
first target update), and reduces each traced copy_ to a box of the
checkpoint tensor and a strided region of the target's storage
(geometry.py). A name is
direct only if every byte it writes is a copy of one incoming byte; anything
else is a fallback, which the engine sends in full through load_weights.
The plan holds for every later update of the same model.
load_planned loads full checkpoint tensors through get_dest; the NCCL
engine uses it. get_boxes (with get_dest) is for engines that send or
read only the slices a rank needs (sharded RDT, NCCL M2N follow-ups).
Classes:
-
LoadPlan–
Functions:
-
check_plannable–Refuse a configuration no load plan supports: a reload mode other than
-
load_planned–Load full checkpoint tensors as
plansays: copy each direct name's -
plan_loads–Trace
model.load_weightsover the declared checkpoint tensors once
LoadPlan
dataclass
¶
Methods:
-
get_boxes–Parts of the checkpoint tensor this rank needs;
None: nothing. -
get_dest–(part of a box, live tensor) pairs: copying each part, reshaped to
Source code in vllm/model_executor/model_loader/reload/plan.py
get_boxes(name)
¶
Parts of the checkpoint tensor this rank needs; None: nothing.
For engines that send or read only those slices (sharded RDT, NCCL
M2N follow-ups); an engine that receives full tensors calls
load_planned instead.
Source code in vllm/model_executor/model_loader/reload/plan.py
get_dest(name)
¶
(part of a box, live tensor) pairs: copying each part, reshaped to
its tensor's shape, into that tensor is exactly what the loader would
have written. None: fallback, send the full tensor through
load_weights. Write only between start_reload and
finish_reload; never read.
load_planned copies full tensors through it; an engine that sends
only get_boxes copies its slices the same way.
Source code in vllm/model_executor/model_loader/reload/plan.py
_copy_all(batch)
¶
One _foreach_copy_ over the batch; on failure, copy pair by pair to
finish it or name the checkpoint tensor that failed.
Source code in vllm/model_executor/model_loader/reload/plan.py
_dest(targets, write, dtype)
¶
The live tensor of write, in the checkpoint dtype.
Source code in vllm/model_executor/model_loader/reload/plan.py
_nbytes(entry)
¶
What the engine moves for entry on this rank.
Source code in vllm/model_executor/model_loader/reload/plan.py
_overlaps(traced, targets)
¶
Names whose writes share a byte with another write: direct writes land
in any order, so these go through load_weights in arrival order.
Source code in vllm/model_executor/model_loader/reload/plan.py
_report(plan, seconds)
¶
Every name at debug; fallbacks grouped by reason and a summary at info.
Source code in vllm/model_executor/model_loader/reload/plan.py
_runs_intersect(ra, rb)
¶
Whether two sorted lists of [start, stop) runs share a byte.
Source code in vllm/model_executor/model_loader/reload/plan.py
_sorted_runs(cache, i, c)
¶
The sorted byte runs of copy c (item i), computed once.
Source code in vllm/model_executor/model_loader/reload/plan.py
_write(copy, shape, dtype, targets)
¶
copy as a box and region; None if it copies nothing.
Raises:
-
Fallback–it is not a plain copy of checkpoint bytes.
Source code in vllm/model_executor/model_loader/reload/plan.py
check_plannable(model)
¶
Refuse a configuration no load plan supports: a reload mode other than
direct, EPLB, and online quantization (its loaders quantize what they
load). Reads only configuration, so an engine can call it
before start_reload changes the model. Direct reload's own refusals
run in begin_direct_reload.
Raises:
-
RuntimeError–the configuration cannot be planned.
Source code in vllm/model_executor/model_loader/reload/plan.py
load_planned(model, plan, weights)
¶
Load full checkpoint tensors as plan says: copy each direct name's
boxes into its destinations, skip names this rank needs nothing of, and
pass fallbacks to model.load_weights. Call between start_reload
and finish_reload.
Copies are batched, so each tensor weights yields must stay unchanged
until this call returns: do not reuse one buffer across items.
Source code in vllm/model_executor/model_loader/reload/plan.py
plan_loads(model, metadata)
¶
Trace model.load_weights over the declared checkpoint tensors once
and return where each lands on this rank.
Call between start_reload and finish_reload: the plan traces
the real tensors in the layout begin_direct_reload applied. Incoming
tensors are data-free traces, so nothing is loaded. Loaders' constant
writes land, as on the load_weights path; so do writes into
non-checkpoint GPU tensors fed by a read of a checkpoint tensor (a
mirror), which are not restored: that name falls back and the update
rewrites them. Module, parameter and buffer state a loader sets stays,
as it would after load_weights.
Raises:
-
ValueError–a name is declared twice.
-
RuntimeError–the model cannot be planned.