GSgnnPerEtypeHitsLPEvaluator
- class graphstorm.eval.GSgnnPerEtypeHitsLPEvaluator(eval_frequency, eval_metric_list=None, major_etype='ALL', use_early_stop=False, early_stop_burnin_rounds=0, early_stop_rounds=3, early_stop_strategy='average_increase')
Bases:
GSgnnBaseEvaluator,GSgnnLPRankingEvalInterface- Evaluator for Link Prediction tasks using
hit@kas metric, and return per edge type
hit@kscores.
Parameters
- eval_frequency: int
The frequency (number of iterations) of doing evaluation.
- eval_metric_list: list of string
Evaluation metric(s) used during evaluation, for example, [“hit_at_10”, “hit_at_100”]. Default: [“hit_at_100”]
- major_etype: tuple
A canonical edge type used for selecting the best model. Default: will use the summation of
hit@kvalues of all edge types.- use_early_stop: bool
Set true to use early stop. Default: False.
- early_stop_burnin_rounds: int
Burn-in rounds (number of evaluations) before starting to check for the early stop condition. Default: 0.
- early_stop_rounds: int
The number of rounds (number of evaluations) for validation scores used to decide early stop. Default: 3.
- early_stop_strategy: str
1)
consecutive_increase, and 2)average_increase. Default:average_increase.
- evaluate(val_rankings, test_rankings, total_iters)
GSgnnLinkPredictionTrainerandGSgnnLinkPredictionInferrerwill call this function to compute validation and testhit@kscores.Parameters
- val_rankings: dict of tensors
Rankings of positive scores of validation edges for each edge type in the format of {etype: ranking}.
- test_rankings: dict of tensors
Rankings of positive scores of test edges for each edge type in the format of {etype: ranking}.
- total_iters: int
The current iteration number.
Returns
- val_score: dict of dict of float
Validation
hit@kscore in the format of {“hit_at_k”: {etype: val_score}}. If theval_rankingis None, return {“hit_at_k”: “N/A”}.- test_score: dict of dict of float
Test
hit@kscore in the format of {“hit_at_k”: {etype: test_score}}. If thetest_rankingis None, return {“hit_at_k”: “N/A”}.
- get_val_score_rank(val_score)
Get the rank of the validation score of the
major_etypeinitialized in class initialization by comparing its value to the existing historical values. If using the defaultmajor_etype, it will compute the rank as the summation of validation values of all edge types.Parameters
- val_score: dict of dict
A dict in the format of {“hit_at_k”: {etype: score}}.
Returns
- rank: int
The rank of the validation score of the given
major_etypeinitialized in class initialization. If using the defaultmajor_etype, the rank will be computed based on the summation of validation scores for all edge types.
- compute_score(rankings, train=True)
Compute per edge type
hit@kevaluation score.Parameters
- rankings: dict of tensors
Rankings of positive scores in the format of {etype: ranking}.
- train: boolean
If in model training.
Returns
- return_metrics: dict of dict of float
Per edge type evaluation
hit@kscore in the format of {“hit_at_k”: {etype: score}}.
- Evaluator for Link Prediction tasks using