GSgnnHitsLPEvaluator

class graphstorm.eval.GSgnnHitsLPEvaluator(eval_frequency, eval_metric_list=None, use_early_stop=False, early_stop_burnin_rounds=0, early_stop_rounds=3, early_stop_strategy='average_increase')

Bases: GSgnnBaseEvaluator, GSgnnLPRankingEvalInterface

Evaluator for Link Prediction tasks using hit@k as metric.

A built-in evaluator for Link Prediction tasks. It uses hit_at_100 as the default eval metric, which implements the GSgnnLPRankingEvalInterface.

Parameters

eval_frequency: int

The frequency (number of iterations) of doing evaluation.

eval_metric_list: list of string

Evaluation metric(s) used during evaluation, for example, [“hit_at_10”, “hit_at_100”]. Default: [“hit_at_100”]

use_early_stop: bool

Set true to use early stop. Default: False.

early_stop_burnin_rounds: int

Burn-in rounds (number of evaluations) before starting to check for the early stop condition. Default: 0.

early_stop_rounds: int

The number of rounds (number of evaluations) for validation scores used to decide early stop. Default: 3.

early_stop_strategy: str

The early stop strategy. GraphStorm supports two strategies: 1) consecutive_increase, and 2) average_increase. Default: average_increase.

evaluate(val_rankings, test_rankings, total_iters)

GSgnnLinkPredictionTrainer and GSgnnLinkPredictionInferrer will call this function to compute validation and test hit@k scores.

Parameters

val_rankings: dict of tensors

Rankings of positive scores of validation edges for each edge type in the format of {etype: ranking}.

test_rankings: dict of tensors

Rankings of positive scores of test edges for each edge type in the format of {etype: ranking}.

total_iters: int

The current iteration number.

Returns

val_score: dict of float

Validation hit@k score in the format of {“hit_at_k”: val_score}. If the val_ranking is None, return {“hit_at_k”: “N/A”}.

test_score: dict of float

Test hit@k score in the format of {“hit_at_k”: test_score}. If the test_ranking is None, return {“hit_at_k”: “N/A”}.

compute_score(rankings, train=True)

Compute hit@k evaluation score.

Parameters

rankings: dict of tensors

Rankings of positive scores in the format of {etype: ranking}

train: boolean

If in model training.

Returns

return_metrics: dict of float

Evaluation hit@k score of in the format of {“hit_at_k”: score}.