mstar.engine.resources.lora.serving

Which LoRA specs a deployment builds from its serving config’s resources: lora: block: the model’s declared specs, sized by the block, narrowed to the targets its adapters touch, and those left with no targets dropped. mstar.model.base.serving_resources adds them to the model’s other specs.

Functions

declared_lora_specs(model, model_config)	The model's LoRA specs, the one keyed lora first, when the YAML has a lora: block; none otherwise.
drop_lora_specs_without_targets(specs)	Leave out a node's LoRA spec whose targets the YAML's targets list dropped entirely: its pool would hold slots for adapters it can add nothing to.
lora_specs(model, model_config)	The deployment's LoRA specs, YAML applied, the one keyed lora first; none when its config turns LoRA on nowhere.
served_lora_targets(lora)	The targets a deployment keeps when its YAML lists no targets: those one of its configured adapters touches, as the targets list every process then applies.
size_lora_specs(lora, model_config)	Apply the YAML's lora: block to the LoRA specs keyed apart from lora.

mstar.engine.resources.lora.serving.declared_lora_specs(model, model_config)

The model’s LoRA specs, the one keyed lora first, when the YAML has a lora: block; none otherwise. The block sizes every spec (one per node that serves adapters): the lora one with the other resources: overrides, the others before them (size_lora_specs), so a block under another LoRA spec’s own key overrides it there.

Parameters:

• model (Model)

• model_config (Mapping[str, Any])

Return type:

list[LoraSpec]

mstar.engine.resources.lora.serving.drop_lora_specs_without_targets(specs)

Leave out a node’s LoRA spec whose targets the YAML’s targets list dropped entirely: its pool would hold slots for adapters it can add nothing to. The modules it owned are then no one’s to load, so the remaining specs refuse an adapter touching them, as a dropped target is refused.

Parameters:

specs (list[NodeResourceSpec])

Return type:

list[NodeResourceSpec]

mstar.engine.resources.lora.serving.lora_specs(model, model_config)

The deployment’s LoRA specs, YAML applied, the one keyed lora first; none when its config turns LoRA on nowhere. Built once per model and resources: block in a process (building one reads the model’s module tree), so its callers there share the specs, and with them a runtime load or unload recorded on one.

Parameters:

• model (Model | None)

• model_config (Mapping[str, Any] | None)

Return type:

list[LoraSpec]

mstar.engine.resources.lora.serving.served_lora_targets(lora)

The targets a deployment keeps when its YAML lists no targets: those one of its configured adapters touches, as the targets list every process then applies. Every other target would hold zero weights in every slot, yet cost its kernels on every step and its memory (the LM head’s and the embedding’s are vocabulary-sized). The first spec (the LLM’s) keeps all of its targets if no adapter touches it; a later spec left with none is not built. None when nothing narrows. The conductor computes it once, before any worker starts, so every process builds the same specs; an adapter that cannot be read fails the deployment here rather than leaving one process with every target.

One list for every spec, as the YAML’s: specs whose target names coincide keep a name either one uses.

Parameters:

lora (list[LoraSpec])

Return type:

list[str] | None

mstar.engine.resources.lora.serving.size_lora_specs(lora, model_config)

Apply the YAML’s lora: block to the LoRA specs keyed apart from lora.

Parameters:

• lora (list[LoraSpec])

• model_config (Mapping[str, Any])

Return type:

None