GSgnnLPRankingEvalInterface
- class graphstorm.eval.GSgnnLPRankingEvalInterface
Bases:
objectInterface for Link Prediction evaluation functions using ranking methods.
The interface sets two abstract methods for Link Prediction evaluator classes that use ranking method to compute evaluation metrics, such as
mrr(Mean Reciprocal Rank).There are two methdos to be implemented if inherite this interface.
1.
evaluate()method, which will be called by different Trainer in theireval()function to provide ranking-based evaluation results of validation and test sets during training process.compute_score()method, which computes the scores for given rankings.
- abstract evaluate(val_rankings, test_rankings, total_iters)
Evaluate Link Prediciton results on validation and test sets.
Link Prediction evaluators should provide the ranking of validation and test sets as input to this method.
Parameters
- val_rankings: dict of tensors
The rankings of validation edges for each edge type in the format of {etype: ranking}.
- test_rankings: dict of tensors
The rankings of testing edges for each edge type in the format of {etype: ranking}.
- total_iters: int
The current iteration number.
Returns
- eval_score: dict
Validation score for each edge type in the format of {etype: score}.
- test_score: dict
Test score for each edge type in the format of {etype: score}.
- abstract compute_score(rankings, train=True)
Compute Link Prediciton evaluation score.
Ranking-based Link Prediction evaluators should provide ranking values as input to this method.
Parameters
- rankings: dict of tensors
Rankings of positive scores in the format of {etype: ranking}
- train: boolean
If in model training.
Returns
dict: Ranking-based evaluation scores for each edge type in the format of {etype: score}.