mstar.engine.resources.sampler.config

What a model declares about sampling: its spec, its per-request config, its step.

Kept free of the resource and its Triton kernels so a submodule can declare a step without pulling them in behind it.

Functions

check_bool_knob(key, value)
check_int_knob(key, value, rule, ok)	Raise ValueError unless value is an integer satisfying ok.
check_real_knob(key, value, rule, ok)	Raise ValueError unless value is a finite number satisfying ok.
derive_seed(seed, stream)	A seed for one named stream of a request, as a non-negative int64.

Classes

LoopbackState(width, dtype, default)	Per-request state a decode loop carries from step to step on the device, beside its last sampled token: one row of width values per request, kept by sampler slot.
SamplerSpec(resource_key, nodes, vocab_size, ...)
SamplerStep(segments, ...] | None = None, ...)
SamplingReqConfig([temperature, top_k, ...])

class mstar.engine.resources.sampler.config.LoopbackState(width, dtype, default)

Bases: object

Per-request state a decode loop carries from step to step on the device, beside its last sampled token: one row of width values per request, kept by sampler slot. A row starts (and restarts, when its slot is reused) at default.

Parameters:

• width (int)

• dtype (dtype)

• default (tuple[int | float, ...])

default: tuple[int | float, ...]

dtype: dtype

width: int

class mstar.engine.resources.sampler.config.SamplerSpec(resource_key: str, nodes: set[str], vocab_size: int | None, enable_repetion_penalty: bool = True, enable_min_p: bool = False, loopback_state: dict[str, mstar.engine.resources.sampler.config.LoopbackState] = <factory>, max_repetition_window: int = 0, min_tokens_stop_ids: tuple[int, ...] = (), enable_top_p_first: bool = False)

Bases: NodeResourceSpec

Parameters:

• resource_key (str)

• nodes (set[str])

• vocab_size (int | None)

• enable_repetion_penalty (bool)

• enable_min_p (bool)

• loopback_state (dict[str, LoopbackState])

• max_repetition_window (int)

• min_tokens_stop_ids (tuple[int, ...])

• enable_top_p_first (bool)

enable_min_p: bool = False

enable_repetion_penalty: bool = True

enable_top_p_first: bool = False

loopback_state: dict[str, LoopbackState]

max_repetition_window: int = 0

min_tokens_stop_ids: tuple[int, ...] = ()

property resource_class: type[Resource]

What builds this spec. Imported inside the property, so declaring a resource stays free of the manager and its kernels.

The builder, not necessarily the class built: an attention spec names AttentionManager, whose build picks a backend subclass.

vocab_size: int | None

class mstar.engine.resources.sampler.config.SamplerStep(segments: tuple[mstar.engine.resources.step.Segment, ...] | None=None, apply_penalty: bool = True, apply_filters: bool = True, prefill_tracked_tokens: dict[str, torch.Tensor]=<factory>, kept_rids: frozenset[int] | None = None, loopback_writes: frozenset[str] = frozenset())

Bases: ResourceStep

Parameters:

• segments (tuple[Segment, ...] | None)

• apply_penalty (bool)

• apply_filters (bool)

• prefill_tracked_tokens (dict[str, Tensor])

• kept_rids (frozenset[int] | None)

• loopback_writes (frozenset[str])

apply_filters: bool = True

apply_penalty: bool = True

kept_rids: frozenset[int] | None = None

loopback_writes: frozenset[str] = frozenset({})

prefill_tracked_tokens: dict[str, Tensor]

class mstar.engine.resources.sampler.config.SamplingReqConfig(temperature: float = 0.6, top_k: int = 0, top_p: float = 1, ignore_eos: bool = False, repetition_penalty: float = 1, min_p: float = 0.0, repetition_window: int = 0, min_tokens: int = 0, top_p_first: bool = False, top_p_min_keep: int = 1, penalize_prompt: bool = True, _seed: int = 0)

Bases: ResourceReqConfig

Parameters:

• temperature (float)

• top_k (int)

• top_p (float)

• ignore_eos (bool)

• repetition_penalty (float)

• min_p (float)

• repetition_window (int)

• min_tokens (int)

• top_p_first (bool)

• top_p_min_keep (int)

• penalize_prompt (bool)

• _seed (int)

apply_conductor_config(seed=0, resource_key=None, **kwargs)

Parameters:

• seed (int)

• resource_key (str | None)

ignore_eos: bool = False

min_p: float = 0.0

min_tokens: int = 0

penalize_prompt: bool = True

repetition_penalty: float = 1

repetition_window: int = 0

property seed

temperature: float = 0.6

top_k: int = 0

top_p: float = 1

top_p_first: bool = False

top_p_min_keep: int = 1

validate(spec=None)

Raise ValueError on a value outside the contract, or one spec’s node cannot honour. None means the sampler’s default.

Parameters:

spec (SamplerSpec | None)

Return type:

None

mstar.engine.resources.sampler.config.check_bool_knob(key, value)

Parameters:

key (str)

Return type:

None

mstar.engine.resources.sampler.config.check_int_knob(key, value, rule, ok)

Raise ValueError unless value is an integer satisfying ok.

Parameters:

• key (str)

• rule (str)

Return type:

None

mstar.engine.resources.sampler.config.check_real_knob(key, value, rule, ok)

Raise ValueError unless value is a finite number satisfying ok.

Parameters:

• key (str)

• rule (str)

Return type:

None

mstar.engine.resources.sampler.config.derive_seed(seed, stream)

A seed for one named stream of a request, as a non-negative int64.

Stable across processes (no hash()), and distinct per stream, so a request’s samplers do not draw correlated numbers from one seed.

Parameters:

• seed (int)

• stream (str)

Return type:

int