GSgnnLPRankingEvalInterface
- class graphstorm.eval.GSgnnLPRankingEvalInterface
Bases:
objectInterface for Link Prediction evaluation functions using ranking methods.
The interface sets two abstract methods for Link Prediction evaluator classes that use ranking method to compute evaluation metrics, such as
mrr(Mean Reciprocal Rank).There are two methods to be implemented if inherit this interface.
1.
evaluate()method, which will be called by different Trainer in theireval()function to provide ranking-based evaluation results of validation and test sets during training process.compute_score()method, which computes the scores for given rankings.
- abstract evaluate(val_rankings, test_rankings, total_iters, **kwargs) Tuple[Dict[str, Tensor], Dict[str, Tensor]]
Evaluate Link Prediction results on validation and test sets.
Link Prediction evaluators should provide the ranking of validation and test sets as input to this method.
Parameters
- val_rankings: dict of tensors
The rankings of validation edges for each edge type in the format of {etype: ranking}.
- test_rankings: dict of tensors
The rankings of testing edges for each edge type in the format of {etype: ranking}.
- total_iters: int
The current iteration number.
- kwargs: dict
Keyword arguments to pass downstream to metric calculation functions.
Returns
- eval_score: dict
Validation score for each edge type in the format of {etype: score}.
- test_score: dict
Test score for each edge type in the format of {etype: score}.
- abstract compute_score(rankings, train=True, **kwargs)
Compute Link Prediction evaluation score.
Ranking-based Link Prediction evaluators should provide ranking values as input to this method.
Parameters
- rankings: dict of tensors
Rankings of positive scores in the format of {etype: ranking}
- train: boolean
If in model training.
- kwargs: dict
Keyword arguments to pass downstream to the metric computation.
Currently we support:
- candidate_sizesdict of tensors
A mapping from edge type to the size of each candidate list (positive + negative pairs). If the tensor has a single element we use that as the size of all lists.
Returns
dict: Ranking-based evaluation scores for each edge type in the format of {etype: score}.