mstar.worker.side_steps

Steps that run beside a worker’s main chain, on a stream of their own.

The main loop runs one chain (CHAIN_LANE): step N on the device, N+1 speculated behind it. A step on that chain costs a pass of the loop and holds the next step behind its kernels on the main stream, which is what a codec window does to the decode steps around it. A node whose node group sets side_stream: true runs its side_stream_walks on a lane of its own instead: the scheduler hands that lane only its keys (MicroScheduler.lane_of), its steps run on the GPU thread between two chain steps, and they land on a second stream while the chain carries on. Submit and postprocess are the chain’s own (Worker._submit_step, Worker._finish_step), keyed by lane.

One step in flight at a time; windows that come ready meanwhile batch into the next one. EngineManager.build decides which node gets the lane and refuses what it can’t serve: a step here is admitted and planned inline, never speculated, planned ahead or lockstep. Its graphs are captured on a stream of their own (Engine._capture_stream_for), which is what makes overlapping them safe.

A resourced node, or a second side node, would each take a lane of their own with an id past this one: admit and plan on the lane’s thread against its own stream, and the same submit, finish and scheduler filter.

Classes

SideStepLane(worker[, lane])

A side lane's stream and the one step in flight on it.

class mstar.worker.side_steps.SideStepLane(worker, lane=SIDE_LANE)

Bases: object

A side lane’s stream and the one step in flight on it.

Parameters:

• worker (Worker)

• lane (int)

execute(batch, node_batch, plan_future)

The GPU thread’s step, on the lane’s stream, with nothing to wait for on the main one: the streamed items it reads had landed before they were routed to it (postprocess waits for the step that made them), and launch collated its windows on this stream.

Parameters:

batch (ScheduledBatch)

finish(block=False)

Postprocess the step in flight once it has landed; block waits.

Never before the step has landed: postprocess frees its inputs first, and those are main-stream allocations a main-stream step could reuse while this one still reads them. A step here holds no resource, so there is no admit to have refused it.

Parameters:

block (bool)

Return type:

None

launch(gpu_executor)

Queue the lane’s next step on the GPU thread, if the stream is free.

Parameters:

gpu_executor (ThreadPool)

Return type:

None