mstar.worker.spec_rows

A speculative step’s per-row objects, built as columns.

Used for a speculation whose continuing rows take all their inputs from the step in flight, which covers a decode loop’s next step. Those rows need no per-row decisions when the step is built, so everything the model and the engine read per row is built here with C-level maps over the runtime’s columns, rather than by a Python loop per row.

Functions

loop_counter_snapshots(infos, loop_name, ...)

Per row, InputMetadata carrying its own copy of the loop counters.

Classes

DeferredRowInputs(request_ids[, signals])

per_request_input_tensors of a speculative step built before the step in flight has produced its outputs.

class mstar.worker.spec_rows.DeferredRowInputs(request_ids, signals=())

Bases: MutableMapping

per_request_input_tensors of a speculative step built before the step in flight has produced its outputs. A row with no inputs of its own maps every signal in signals to an empty list, built the first time the row is read.

When every continuing row’s inputs stayed on the device, threading N’s outputs in only names the signals (set_signals), and a step the engine prepares through uniform_row_inputs never builds a row. Otherwise threading materialize``s the rows and splices them one by one. Fresh rows are given the inputs their pop gathered (``rows[rid] = inputs), which set_signals leaves alone.

Parameters:

• request_ids (Iterable[int])

• signals (Iterable[str])

materialize()

A plain dict of every row, for a path that edits rows one by one.

Return type:

dict[int, dict[str, list[Tensor]]]

set_signals(signals)

Every template row now carries signals, each with no tensors, including a row already read.

Parameters:

signals (Iterable[str])

Return type:

None

mstar.worker.spec_rows.loop_counter_snapshots(infos, loop_name, n_next_iter)

Per row, InputMetadata carrying its own copy of the loop counters.

The first n_next_iter rows run one iteration past the shared counters in loop_name; the rest run at them. A copy, not a view: the shared counters move when the in-flight step is routed, which can happen while this step is being prepared.

Parameters:

• infos (list[CurrentForwardPassInfo])

• loop_name (str | None)

• n_next_iter (int)

Return type:

list[InputMetadata]