gigl.distributed.dist_sampling_producer#

Attributes#

logger

Classes#

DistSamplingProducer

A subprocess group of distributed sampling workers.

Module Contents#

class gigl.distributed.dist_sampling_producer.DistSamplingProducer(data, sampler_input, sampling_config, worker_options, channel, sampler_options, degree_tensors=None)[source]#

Bases: graphlearn_torch.distributed.DistMpSamplingProducer

A subprocess group of distributed sampling workers.

Note that this producer is only used for workload with separate sampling and training, all sampled results will be sent to the output channel.

Parameters:

  • data (graphlearn_torch.distributed.DistDataset)

  • sampler_input (Union[graphlearn_torch.sampler.NodeSamplerInput, graphlearn_torch.sampler.EdgeSamplerInput])

  • sampling_config (graphlearn_torch.sampler.SamplingConfig)

  • worker_options (graphlearn_torch.distributed.MpDistSamplingWorkerOptions)

  • channel (graphlearn_torch.channel.ChannelBase)

  • sampler_options (gigl.distributed.sampler_options.SamplerOptions)

  • degree_tensors (Optional[Union[torch.Tensor, dict[graphlearn_torch.typing.NodeType, torch.Tensor]]])

init()[source]#

Create the subprocess pool. Init samplers and rpc server.

shutdown()[source]#

Stops the sampling workers, waiting at most MP_STATUS_CHECK_INTERVAL in total.

Same as GLT’s DistMpSamplingProducer.shutdown [1] except that the grace period is a single deadline shared by all workers. See join_workers_with_shared_deadline().

[1]: alibaba/graphlearn-for-pytorch

Return type:

None

gigl.distributed.dist_sampling_producer.logger[source]#