gigl.src.training.v1.lib.data_loaders.utils#

Attributes#

logger

Functions#

`cast_graph_for_training`(batch_graph_data, ...)	Casts the PygGraphData object into a Data or HeteroData object. Also fills in any missing fields from graph
`get_data_split_for_current_worker`(data_list)	Split list of data per worker

Module Contents#

gigl.src.training.v1.lib.data_loaders.utils.cast_graph_for_training(batch_graph_data, graph_metadata_pb_wrapper, preprocessed_metadata_pb_wrapper, batch_type, should_register_edge_features)[source]#

Casts the PygGraphData object into a Data or HeteroData object. Also fills in any missing fields from graph builder with empty tensors in cases where there are no edges for a graph or given edge type. :param batch_graph_data: Coalesced batch graph :type batch_graph_data: PygGraphData :param graph_metadata_pb_wrapper: Graph Metadata Pb Wrapper for this training job :type graph_metadata_pb_wrapper: GraphMetadataPbWrapper :param preprocessed_metadata_pb_wrapper: Preprocessed Metadata Pb Wrapper for this training job :type preprocessed_metadata_pb_wrapper: PreprocessedMetadataPbWrapper :param should_register_edge_features: Whether we should register edge features for the built graph :type should_register_edge_features: bool

Parameters:

batch_graph_data (gigl.src.common.graph_builder.pyg_graph_data.PygGraphData)
graph_metadata_pb_wrapper (gigl.src.common.types.pb_wrappers.graph_metadata.GraphMetadataPbWrapper)
preprocessed_metadata_pb_wrapper (gigl.src.common.types.pb_wrappers.preprocessed_metadata.PreprocessedMetadataPbWrapper)
batch_type (str)
should_register_edge_features (Optional[bool])

Return type:

Union[torch_geometric.data.Data, torch_geometric.data.hetero_data.HeteroData]

gigl.src.training.v1.lib.data_loaders.utils.get_data_split_for_current_worker(data_list)[source]#

Split list of data per worker Selects a subset of data based on Torch get_worker_info. Used as a shard selection function in Dataset.

Parameters:: data_list (numpy.ndarray)
Return type:: numpy.ndarray

gigl.src.training.v1.lib.data_loaders.utils.logger[source]#