gigl.src.common.models.layers.loss#

Classes#

`AligmentLoss`	Taken from THUwangcy/DirectAU, AlignmentLoss increases the similarity of representations between positive user-item pairs.
`BGRLLoss`	Leverages BGRL loss from https://arxiv.org/pdf/2102.06514.pdf, using an offline and online encoder to predict alternative augmentations of
`FeatureReconstructionLoss`	Computes SCE between original feature and reconstructed feature. See https://arxiv.org/pdf/2205.10803.pdf for more information about
`GBTLoss`	Computes the Barlow Twins loss on the two input matrices as an auxiliary loss.
`GRACELoss`	A loss class that implements the GRACE (https://arxiv.org/pdf/2006.04131.pdf) contrastive loss approach. We generate two graph views by
`KLLoss`	Calculates KL Divergence between two set of scores for the distribution loss.
`LLPRankingLoss`	Calculates a margin-based rakning loss between two set of scores for the ranking loss in LLP.
`MarginLoss`	A loss layer built on top of the PyTorch implementation of the margin ranking loss.
`ModelResultType`	Generic enumeration.
`RetrievalLoss`	A loss layer built on top of the tensorflow_recommenders implementation.
`SoftmaxLoss`	A loss layer built on top of the PyTorch implementation of the softmax cross entropy loss.
`TBGRLLoss`	TBGRL (https://arxiv.org/pdf/2211.14394.pdf) improves over BGRL by generating a third augmented graph as a negative sample,
`UniformityLoss`	Taken from THUwangcy/DirectAU, UniformityLoss measures how well the representations scatter on the hypersphere.
`WhiteningDecorrelationLoss`	Utilizes canonical correlation analysis to compute similarity between augmented graphs as an auxiliary loss. See https://arxiv.org/pdf/2106.12484.pdf

Module Contents#

class gigl.src.common.models.layers.loss.AligmentLoss(alpha=2.0)[source]#

Bases: torch.nn.Module

Taken from THUwangcy/DirectAU, AlignmentLoss increases the similarity of representations between positive user-item pairs.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:: alpha (Optional[float])

forward(user_embeddings, item_embeddings)[source]#

Parameters:

user_embeddings (torch.Tensor)
item_embeddings (torch.Tensor)

Return type:

torch.Tensor

alpha = 2.0[source]#

class gigl.src.common.models.layers.loss.BGRLLoss(*args, **kwargs)[source]#

Bases: torch.nn.Module

Leverages BGRL loss from https://arxiv.org/pdf/2102.06514.pdf, using an offline and online encoder to predict alternative augmentations of the input. The offline encoder is updated by an exponential moving average rather than traditional backpropogation. We use BGRL as an auxiliary loss for improved generalization.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(q1, q2, y1, y2)[source]#

Parameters:

q1 (torch.Tensor)
q2 (torch.Tensor)
y1 (torch.Tensor)
y2 (torch.Tensor)

Return type:

Tuple[torch.Tensor, int]

class gigl.src.common.models.layers.loss.FeatureReconstructionLoss(alpha=3.0)[source]#

Bases: torch.nn.Module

Computes SCE between original feature and reconstructed feature. See https://arxiv.org/pdf/2205.10803.pdf for more information about feature reconstruction. We use this as an auxiliary loss for training and improved generalization.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:: alpha (float)

forward(x_target, x_pred)[source]#

Parameters:

x_target (torch.Tensor)
x_pred (torch.Tensor)

Return type:

Tuple[torch.Tensor, int]

alpha = 3.0[source]#

class gigl.src.common.models.layers.loss.GBTLoss[source]#

Bases: torch.nn.Module

Computes the Barlow Twins loss on the two input matrices as an auxiliary loss. From the offical GBT implementation at: pbielak/graph-barlow-twins

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(z_a, z_b, device)[source]#

Parameters:

z_a (torch.Tensor) – First input matrix
z_b (torch.Tensor) – Second input matrix
device (torch.device) – the device to set as default

Returns:

The Barlow Twins loss and the sample size

Return type:

Tuple[torch.Tensor, int]

eps = 1e-15[source]#

class gigl.src.common.models.layers.loss.GRACELoss(temperature=None)[source]#

Bases: torch.nn.Module

A loss class that implements the GRACE (https://arxiv.org/pdf/2006.04131.pdf) contrastive loss approach. We generate two graph views by corruption and learn node representations by maximizing the agreement of node representations in these two views. We introduce this to add an additional contrastive loss function for multi-task learning.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:: temperature (Optional[float])

forward(h1, h2, device=torch.device('cpu'))[source]#

Parameters:

h1 (torch.Tensor) – First input tensor
h2 (torch.Tensor) – Second input tensor
device (torch.device) – the device to set as default

Returns:

The loss and the sample size

Return type:

Tuple[torch.Tensor, int]

temperature = None[source]#

class gigl.src.common.models.layers.loss.KLLoss(kl_temperature)[source]#

Bases: torch.nn.Module

Calculates KL Divergence between two set of scores for the distribution loss. Taken from: snap-research/linkless-link-prediction

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:: kl_temperature (float)

forward(student_scores, teacher_scores)[source]#

Parameters:

student_scores (torch.Tensor)
teacher_scores (torch.Tensor)

Return type:

torch.Tensor

kl_temperature[source]#

class gigl.src.common.models.layers.loss.LLPRankingLoss(margin)[source]#

Bases: torch.nn.Module

Calculates a margin-based rakning loss between two set of scores for the ranking loss in LLP. This differs from normal margin loss in that it prevents the student model from trying to differentiate miniscule differences in probabilities which the teacher may make w/ due to noise. Taken from: snap-research/linkless-link-prediction

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:: margin (float)

forward(student_scores, teacher_scores, device)[source]#

Parameters:

student_scores (torch.Tensor)
teacher_scores (torch.Tensor)
device (torch.device)

Return type:

torch.Tensor

margin[source]#

margin_loss[source]#

class gigl.src.common.models.layers.loss.MarginLoss(margin=None)[source]#

Bases: torch.nn.Module

A loss layer built on top of the PyTorch implementation of the margin ranking loss.

The loss function by default calculates the loss by: margin_ranking_loss(pos_scores, hard_neg_scores, random_neg_scores, margin=margin, reduction=’sum’)

It encourages the model to generate higher similarity scores for positive pairs than negative pairs by at least a margin.

See: https://pytorch.org/docs/stable/generated/torch.nn.MarginRankingLoss.html for more information.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:: margin (Optional[float])

forward(loss_input, device=torch.device('cpu'))[source]#

Parameters:

loss_input (List[Dict[gigl.src.common.types.graph_data.CondensedEdgeType, gigl.src.common.types.task_inputs.BatchScores]])
device (torch.device)

Return type:

Tuple[torch.Tensor, int]

margin = None[source]#

class gigl.src.common.models.layers.loss.ModelResultType[source]#

Bases: enum.Enum

Generic enumeration.

Derive from this class to define new enumerations.

batch_combined_scores = 'batch_combined_scores'[source]#

batch_embeddings = 'batch_embeddings'[source]#

batch_scores = 'batch_scores'[source]#

input_batch = 'input_batch'[source]#

class gigl.src.common.models.layers.loss.RetrievalLoss(loss=None, temperature=None, remove_accidental_hits=False)[source]#

Bases: torch.nn.Module

A loss layer built on top of the tensorflow_recommenders implementation. https://www.tensorflow.org/recommenders/api_docs/python/tfrs/tasks/Retrieval

The loss function by default calculates the loss by: ` cross_entropy(torch.mm(query_embeddings, candidate_embeddings.T), positive_indices, reduction='sum'), ` where the candidate embeddings are torch.cat((positive_embeddings, random_negative_embeddings)). It encourages the model to generate query embeddings that yield the highest similarity score with their own first hop compared with others’ first hops and random negatives. We also filter out the cases where, in some rows, the query could accidentally treat its own positives as negatives.

Parameters:

loss (Optional[nn.Module]) – Custom loss function to be used. If None, the default is nn.CrossEntropyLoss(reduction=”sum”).
temperature (Optional[float]) – Temperature scaling applied to scores before computing cross-entropy loss. If not None, scores are divided by the temperature value.
remove_accidental_hits (bool) – Whether to remove accidental hits where the query’s positive items are also present in the negative samples.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

calculate_batch_retrieval_loss(scores, candidate_sampling_probability=None, query_ids=None, candidate_ids=None, device=torch.device('cpu'))[source]#

Parameters:

scores (torch.Tensor) – [num_queries, num_candidates] tensor of candidate and query embeddings similarity
candidate_sampling_probability (Optional[torch.Tensor]) – [num_candidates], Optional tensor of candidate sampling probabilities. When given will be used to correct the logits toreflect the sampling probability of negative candidates.
query_ids (Optional[torch.Tensor]) – [num_queries] Optional tensor containing query ids / anchor node ids.
candidate_ids (Optional[torch.Tensor]) – [num_candidates] Optional tensor containing candidate ids.
device (torch.device) – the device to set as default

Return type:

torch.Tensor

forward(batch_combined_scores, repeated_query_embeddings, candidate_sampling_probability, device=torch.device('cpu'))[source]#

Parameters:

batch_combined_scores (gigl.src.common.types.task_inputs.BatchCombinedScores)
repeated_query_embeddings (torch.FloatTensor)
candidate_sampling_probability (Optional[torch.FloatTensor])
device (torch.device)

Return type:

Tuple[torch.Tensor, int]

class gigl.src.common.models.layers.loss.SoftmaxLoss(softmax_temperature=None)[source]#

Bases: torch.nn.Module

A loss layer built on top of the PyTorch implementation of the softmax cross entropy loss.

The loss function by default calculate the loss by: cross_entropy(all_scores, ys, reduction=’sum’)

See: https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html for more information.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:: softmax_temperature (Optional[float])

forward(loss_input, device=torch.device('cpu'))[source]#

Parameters:

loss_input (List[Dict[gigl.src.common.types.graph_data.CondensedEdgeType, gigl.src.common.types.task_inputs.BatchScores]])
device (torch.device)

Return type:

Tuple[torch.Tensor, int]

softmax_temperature = None[source]#

class gigl.src.common.models.layers.loss.TBGRLLoss(neg_lambda=0.12)[source]#

Bases: torch.nn.Module

TBGRL (https://arxiv.org/pdf/2211.14394.pdf) improves over BGRL by generating a third augmented graph as a negative sample, providing a cheap corruption that improves generalizability of the model in inductive settings. We use TBGRL as an auxiliary loss for improved generalization.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:: neg_lambda (float)

forward(q1, q2, y1, y2, neg_y)[source]#

Parameters:

q1 (torch.Tensor)
q2 (torch.Tensor)
y1 (torch.Tensor)
y2 (torch.Tensor)
neg_y (Optional[torch.Tensor])

Return type:

Tuple[torch.Tensor, int]

neg_lambda = 0.12[source]#

class gigl.src.common.models.layers.loss.UniformityLoss(temperature=2.0)[source]#

Bases: torch.nn.Module

Taken from THUwangcy/DirectAU, UniformityLoss measures how well the representations scatter on the hypersphere.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:: temperature (float)

forward(user_embeddings, item_embeddings)[source]#

Parameters:

user_embeddings (torch.Tensor)
item_embeddings (torch.Tensor)

Return type:

torch.Tensor

temperature = 2.0[source]#

class gigl.src.common.models.layers.loss.WhiteningDecorrelationLoss(lambd=0.001)[source]#

Bases: torch.nn.Module

Utilizes canonical correlation analysis to compute similarity between augmented graphs as an auxiliary loss. See https://arxiv.org/pdf/2106.12484.pdf for more information.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:: lambd (float)

forward(h1, h2, N, device=torch.device('cpu'))[source]#

Parameters:

h1 (torch.Tensor) – First input tensor
h2 (torch.Tensor) – Second input tensor
N (int) – The number of samples
device (torch.device) – the device to set as default

Returns:

The loss and the sample size

Return type:

Tuple[torch.Tensor, int]

lambd = 0.001[source]#