gigl.distributed.utils.channel#

Classes#

MonitoredShmChannel

Extends ShmChannel with queue size method qsize() by attaching to the channels memory region and inspecting the C++ struct layout.

SizedShmChannel

Extends ShmChannel with queue size method qsize() by attaching to the channels memory region and inspecting the C++ struct layout.

Module Contents#

class gigl.distributed.utils.channel.MonitoredShmChannel(channel_name, *args, **kwargs)[source]#

Bases: SizedShmChannel

Extends ShmChannel with queue size method qsize() by attaching to the channels memory region and inspecting the C++ struct layout.

TODO: Revisit direct memory inspection vs. custom C++ channel implementation. Current solution inspects GLT’s underlying C++ memory layout for simplicity, avoiding the overhead of porting GLT’s full C++ queue code into GiGL. Revisit and implement a native C++ channel with public size methods if:

  1. Deeper channel monitoring is needed (e.g., % queue filled in bytes).

  2. We roll a custom IPC queue for other architectural reasons.

  3. An upstream GLT release breaks the struct memory layout (tests should catch this).

Monitored variant of SizedShmChannel that integrates with GiGL metrics service and records queue size on recv() as a gauge.

Parameters:

  • channel_name (str) – Prefix for published metrics. Must be unique across processes to disambiguate channels owned by different dataloaders. Multiple instances within the same process are automatically disambiguated by an internally appended sequence ID (e.g., id0).

  • *args – Positional arguments forwarded directly to ShmChannel.

  • **kwargs – Keyword arguments forwarded directly to ShmChannel.

Example

Passing channel_name=”my_channel_pid_12345” publishes queue size for the first instance as my_channel_pid_12345_id0_qsize.

recv(*args, **kwargs)[source]#

Recv a sample message from channel.

Return type:

graphlearn_torch.channel.SampleMessage

class gigl.distributed.utils.channel.SizedShmChannel(*args, **kwargs)[source]#

Bases: graphlearn_torch.channel.ShmChannel

Extends ShmChannel with queue size method qsize() by attaching to the channels memory region and inspecting the C++ struct layout.

TODO: Revisit direct memory inspection vs. custom C++ channel implementation. Current solution inspects GLT’s underlying C++ memory layout for simplicity, avoiding the overhead of porting GLT’s full C++ queue code into GiGL. Revisit and implement a native C++ channel with public size methods if:

  1. Deeper channel monitoring is needed (e.g., % queue filled in bytes).

  2. We roll a custom IPC queue for other architectural reasons.

  3. An upstream GLT release breaks the struct memory layout (tests should catch this).

qsize()[source]#

The number of SampleMessage items currently in the channel.

Return type:

int